Non-Relational Databases in System Design: How to Actually Pick the Right One

In the previous post, we looked at relational databases — their strong guarantees, ACID transactions, and how to scale them with replicas and sharding. This time, we turn to the other big family: non-relational databases, often called NoSQL databases.

Non-relational databases are enormously popular, and for good reason. But they’re also widely misunderstood. Many teams reach for them for the wrong reasons and end up with a system that’s harder to work with, not easier. So in this post, we’ll cover what these databases are, why they scale so well, and — most importantly — how to pick the right database for your system instead of guessing.

The Big Advantage: Sharding Out of the Box

The first thing to know about most non-relational databases is that they shard out of the box. In the relational world, sharding — splitting your data across multiple machines — is something you usually have to design and manage yourself. It’s powerful, but it takes real effort.

Non-relational databases flip this around. Sharding is built into the way they work. You give the database more machines, and it spreads the data across them for you, largely automatically. This is a big part of why they’ve earned a reputation for scaling easily.

The Three Main Types

Undestand Non-Relational Database

“Non-relational” is a broad umbrella. Under it sit several distinct types, each suited to a different job. The three most common are these.

Document databases store data as flexible documents, usually in a JSON-like format. Each document can hold nested fields and can differ in shape from the next. MongoDB is the classic example. These are a natural fit when your data is object-like — for instance, a user profile with an address, a list of hobbies, and settings, all stored together in one document.

Key-value stores are the simplest model: every piece of data is stored against a unique key, like a giant dictionary. You hand over a key, and you get the value back — extremely fast. Redis is a well-known example. These shine for things like caching, session storage, or leaderboards where you look data up by a known key.

Graph databases are built for data that is all about relationships. Instead of rows and tables, they store nodes (entities) and edges (the connections between them). Neo4j is a popular example. Think of a social network: people are nodes, friendships are edges, and questions like “who are the friends of my friends?” become natural to answer.

The Common Misconception: “Relational Databases Don’t Scale”

Here’s one of the most widespread misunderstandings in system design: teams choose a non-relational database because they believe relational databases don’t scale.

This reasoning is flawed. As we saw in the previous post, relational databases can scale — with read replicas, vertical scaling, and sharding, they power some of the largest systems in the world. Scaling is not the reason relational databases fall short, and it shouldn’t be the sole reason you abandon them.

To choose well, you need to understand why non-relational databases scale so easily in the first place. And the answer might surprise you: they scale easily precisely because they give things up.

Why Non-Relational Databases Scale So Easily

Non-Relational Database

Non-relational databases scale well not through magic, but through trade-offs. They deliberately drop some of the features that make relational databases hard to distribute.

No rigid structure. Relational databases enforce a fixed schema — every row in a table must match the defined columns. Non-relational databases relax this. Documents can vary in shape, which makes them easier to spread across machines without coordinating a shared structure.

No relations across the data. Relational databases let any table reference any other, and keeping those references valid across many machines is genuinely hard. Non-relational databases largely avoid cross-record relationships, so there’s far less to coordinate when data is split up.

No constraints to enforce globally. Constraints like uniqueness and foreign keys require the database to check rules across the whole dataset. If your data lives on twenty machines, enforcing a global rule means those machines must constantly talk to each other. Non-relational databases skip most of these global constraints, removing that coordination cost.

Put simply: the data is modeled to be sharded. Because each record is self-contained and doesn’t depend on records elsewhere, the database can place it on any machine without worrying about the rest. There’s nothing to join, nothing to cross-check, nothing to keep globally consistent. That independence is exactly what makes horizontal scaling smooth.

The trade-off, of course, is that you lose those very features — the joins, the constraints, the strong guarantees. Which is fine if you didn’t need them. That’s the whole point.

How This Shapes System Design

Understanding why these databases scale changes how you design a system. Instead of asking “which database is trendy?”, you start asking “does my data actually depend on those relational features, or can I model it to be independent?”

If your data naturally breaks into self-contained pieces — a product catalog, user sessions, event logs — then a non-relational database lets you scale almost effortlessly, because the data was shardable to begin with. But if your data is deeply interconnected and correctness across records matters, giving up relations and constraints will hurt you far more than it helps. Knowing the trade-off lets you design around it deliberately, rather than discovering the pain later.

Don’t Jump Straight to a Database

The biggest mistake engineers make is picking a specific database first and figuring out the requirements later. It should be the other way around. Before you name a single product, understand what you actually need. Ask yourself these questions.

What data are you storing? Is it object-like documents, simple key-value pairs, deeply connected relationships, or structured records with clear columns?

How much data will you store? Are we talking a few gigabytes that fit comfortably on one machine, or terabytes that will never fit on a single node?

How will you access the data? Will you mostly look things up by a known key, or will you run rich, flexible queries across many fields?

What kind of queries will you fire? Simple lookups? Heavy aggregations and reporting? Multi-step graph traversals?

Do you need any special features? Some databases offer capabilities that can dramatically simplify your design, such as:

• Expiry (TTL) — data that automatically deletes itself after a set time, perfect for caches and sessions.
• Bloom filters — a memory-efficient way to check whether an item is probably present or definitely absent, useful for skipping expensive lookups.
• HyperLogLog — a clever structure for estimating the count of unique items (like unique visitors) using very little memory.

Only once you’ve answered these questions are you ready to choose. The right database falls out of the requirements, not the other way around.

How to Actually Pick the Right Database

With those requirements in hand, choosing becomes much clearer. Here’s a practical decision guide based on common scenarios.

Your data fits on a single node, and you need strong consistency, correctness, and complex queries or aggregations → Relational database. If your dataset is manageable in size and you rely on joins, transactions, and analytical queries, a relational database like PostgreSQL is the right home. Don’t walk away from it just because “NoSQL is popular.”

Your access is key-value based, you need it to be extremely fast, and you want advanced data structures → Redis. When you look data up by a known key and speed is everything — caching, rate limiting, leaderboards — an in-memory key-value store like Redis is ideal. It also offers rich built-in structures (lists, sets, sorted sets) that can replace a lot of custom code.

Your data doesn’t fit on one node → a non-relational database (such as a document store). Once your dataset outgrows what a single machine can hold, a database that shards out of the box, like MongoDB, saves you enormous operational effort. It’s built to spread data across many machines from day one.

You need sophisticated graph algorithms → Graph database. If your core problem is about relationships and traversals — shortest paths, recommendations, fraud rings, “friends of friends” — a graph database like Neo4j is purpose-built for it, and will run circles around trying to force this into a relational or document model.

You don’t have anything specific in mind and want to stay flexible for an unknown future → a document database like MongoDB. When requirements are still fuzzy, and you’re not sure what your data will look like in six months, a flexible-schema document store lets you adapt as you learn, without locking you into rigid decisions too early.

A Quick Note on the “Doesn’t Fit on One Node” Rule

It’s worth being precise here, because this point is easy to muddle. If your data fits on a single node and you value consistency and rich queries, lean relational. If your data cannot fit on a single node, that’s a strong signal to reach for a non-relational database that shards naturally. The size of your data — whether it lives on one machine or must span many — is one of the clearest deciding factors of all.

Wrapping Up

Non-relational databases are powerful, but they aren’t a free upgrade over relational ones. They scale so gracefully precisely because they give up structure, relations, and constraints — modeling data to be independent and therefore shardable. That’s a wonderful trade when you don’t need those features, and a painful one when you do.

So resist the urge to pick a database first. Start with your data: what it looks like, how much of it there is, how you’ll access it, and what special features would make your life easier. Let those answers point you to the right tool — whether that’s PostgreSQL, Redis, MongoDB, a graph database, or something else entirely.

Choose the database that fits your problem, not the one that fits the trend. That single discipline will save you more pain than almost any other decision in system design.