Lifecycle of query in MongoDB Sharded Cluster

The life cycle of a query in a MongoDB sharded cluster involves several stages as it’s processed through the cluster. It is recommended that you first check out this article in order to get to know about basics of sharded mongo cluster first. Here’s a step-by-step explanation of the query life cycle in a sharded cluster:

Client Request

• The life cycle begins when a client application sends a read or write request to the MongoDB cluster. The request is usually sent to one of the “mongos” processes, which are the query routers in the cluster.

“mongos” Processing

• The client request is received by one of the “mongos” query routers.
• The “mongos” process examines the request and its associated shard key, if applicable.
• If the query contains a shard key and is a read operation, the “mongos” uses the shard key to determine the shard(s) or chunk(s) involved in the query. If it’s a write operation, the “mongos” identifies the shard where the data should be written.

Routing

• For read operations, if the query can be satisfied by a single shard (using the shard key), the “mongos” routes the query directly to that shard. This is called a targeted query and is the most efficient scenario.

Routers parllelizes request to all relevant shards

• For read operations that involve multiple shards (scatter-gather query), the “mongos” parallelizes the request to all relevant shards and then merges the results before returning them to the client.
• For write operations, the “mongos” routes the request to the appropriate shard, which will be responsible for handling the write.

Shard Processing

• When the query reaches the designated shard(s), they execute the read or write operation as per the request.
• For read operations, the shard returns the requested data or results of the query.
• For write operations, the shard writes the data and acknowledges the write.

Aggregation (Optional)

• If the client query includes aggregation, data transformation, or complex operations, they are performed on the shards where the data resides.

Chunk Migration (If Needed)

• In the case of a read or write operation that involves data distributed across multiple shards, the “mongos” may need to coordinate the scatter-gather operation. The “mongos” sends the queries to relevant shards and merges the results.
• If the balancer has determined that data should be moved from one shard to another for better distribution, chunk migrations may occur as a background process. The balancer manages these migrations, and the data is transferred between shards without the client being involved.

Result Aggregation

• For read operations that were parallelized to multiple shards, the “mongos” aggregates the results from all shards and returns the combined results to the client.

Response to Client

• Finally, the “mongos” process sends the query results or acknowledgments for write operations back to the client application that initiated the query.

The query life cycle in a MongoDB sharded cluster demonstrates the complexities and optimizations involved in routing, executing, and aggregating data across distributed shards. Proper configuration and shard key selection are important factors in ensuring efficient query routing and distribution in a sharded cluster.

This article is based on learning from talk of Sanika Phanse from Mongo Sharding Team — NYC, Check out the talk here:

https://youtu.be/EvzPncoCr_M