Designing Scalable Systems with the Scatter-Gather Pattern

One of the most common architecture patterns is the Scatter-Gather pattern. Before diving into it, let’s look at an example without using it to understand the problem.

If we have a hotel booking system where users search and compare rooms across a large number of hotels to find the best option, one possible design is for the backend service to call external hotel APIs sequentially (each API call waits for the previous one to finish). Once all responses are received, the backend builds the final result and sends it to the user.

Problem

The problem with this design is that the request takes a very long time to return to the user due to sequential searching. The total request time is approximately the sum of all external hotel API response times.

The Problem — Sequential API Calls

How does the Scatter-Gather pattern fix this problem?

Instead of calling external APIs sequentially, the request is sent to a dispatcher. The dispatcher then scatters the request to multiple workers, and each worker is responsible for querying one or more data sources, getting the result, and sending it back to the dispatcher. The dispatcher gathers and aggregates the results before sending the final response to the user.

With this approach, the total response time becomes approximately the slowest worker response time rather than the sum of all response times, plus a small amount of dispatching and aggregation overhead.

It’s important to note that Scatter-Gather is not limited to parallel execution within a single process (for example, using “Promise.all”). It is commonly used in distributed systems where requests are sent to multiple independent services or data sources and their responses are aggregated into a single result.

However, there are some cases that must be handled when using this pattern. For example, one or more workers may take a long time to respond or become unavailable. To handle this, the dispatcher should be configured with a maximum wait time. If this time is exceeded, late workers are ignored and the dispatcher returns a partial result using the responses received before the timeout expires.

Another important consideration is correlation between requests and responses. Each request should contain a correlation ID, and workers should include this ID in their responses so the dispatcher can aggregate only the responses that belong to the same user request.

Decoupling the Dispatcher and Workers

In the above design, the dispatcher and workers are tightly coupled because the dispatcher knows the address and count of each worker. Instead of using HTTP or gRPC for communication between the dispatcher and workers, a message broker can be used.

The Decoupled Pattern — Using a Message Broker

When using a message broker, the dispatcher publishes a request message, and the broker distributes it to the appropriate workers according to the configured routing strategy. Once a worker finishes processing, it publishes the result to a queue or topic that the dispatcher listens on. When the maximum wait time is exceeded, the dispatcher aggregates the available results and sends the final response to the user.

What are workers?

Workers are not limited to calling external APIs. They may be instances of the same service looking at different data partitions, services responsible for different data sources, or services that perform different types of processing before returning their results.

Conclusion

The Scatter-Gather pattern is very useful for systems that need to query multiple sources in parallel and aggregate their responses into a single result, such as search aggregators, travel booking platforms, recommendation engines, and data aggregation services.

Like any architecture pattern, it adds complexity to the system, so before using it you must consider the trade-offs and determine whether the performance benefits justify the additional complexity for your specific use case.