CAP Theorem: The Thing Everyone References and Nobody Fully Understands

I was in an architecture review when someone asked, “Is this system CP or AP?” The room went quiet. Then three different people gave three different answers — and they were all technically wrong. This is the CAP theorem experience. It’s the distributed systems equivalent of everyone nodding along to jazz — we all pretend we get it.

The CAP theorem might be the most misunderstood concept in distributed systems. It’s cited constantly, usually incorrectly, and has spawned an entire generation of engineers who think they understand distributed systems trade-offs when they’ve actually just memorized a misleading oversimplification.

Let me share what I’ve learned from watching architects actually wrestle with these trade-offs — and why the “pick two of three” framing you’ve probably heard is doing you more harm than good.

The Original Idea (And Why It Mattered)

In 2000, Eric Brewer presented a conjecture at the ACM Symposium on Principles of Distributed Computing. Two years later, Seth Gilbert and Nancy Lynch proved it as a theorem. The basic claim:

Any networked shared-data system can have at most two of three desirable properties: Consistency, Availability, and Partition Tolerance.

The theorem served its purpose: it opened designers’ minds to a wider range of systems and trade-offs. Before CAP, the database world was dominated by ACID-compliant systems that prioritized consistency above all else. CAP gave permission to explore alternatives — and the NoSQL movement ran with it like a toddler with scissors.

But here’s what Brewer himself said twelve years later: “The ‘2 of 3’ formulation was always misleading because it tended to oversimplify the tensions among properties.”

When the author of the theorem says you’re using it wrong, maybe it’s time to listen. Let me unpack why.

What CAP Actually Means (The Precise Definitions)

If you want to use CAP as a theorem (not just a vague concept), you need to use the words exactly as the proof defines them. And those definitions are narrower than most people realize. This is where the overengineering of understanding begins — and paradoxically, where most people under-engineer their understanding.

Consistency © = Linearizability

CAP’s “consistency” doesn’t mean what you think it means. It’s not the C in ACID. It’s not “data integrity.” It’s a very specific property called linearizability.

Martin Kleppmann, author of Designing Data-Intensive Applications, explains it this way: “If operation B started after operation A successfully completed, then operation B must see the system in the same state as it was on completion of operation A, or a newer state.”

Here’s a concrete example of a non-linearizable system:

Bob hit refresh after hearing Alice’s result, so he expected at least as recent data. Getting stale data violates linearizability — even though both requests “succeeded.” Bob is now questioning reality, his friendship with Alice, and the entire concept of distributed systems.

This is an expensive guarantee. Even your CPU doesn’t provide linearizable access to local RAM by default — you need explicit memory barrier instructions. Providing linearizability across a distributed system requires significant coordination.

Availability (A) = Every Non-Failing Node Responds

CAP’s availability is stricter than “the system is up.” The formal definition: “Every request received by a non-failing node in the system must result in a non-error response.”

Notice what this means: it’s not enough for some node to handle the request. Any non-failing node must be able to handle it. Many systems marketed as “highly available” don’t actually meet this definition. Marketing and formal proofs have a complicated relationship.

Also note: CAP-available systems are allowed to be arbitrarily slow. A system that takes two minutes to respond is still “available” under CAP. Your users would disagree. Your SLA would disagree. Your on-call engineer would strongly disagree.

Partition Tolerance (P) = Asynchronous Network

This is the most misunderstood property. “Partition tolerance” is a terrible name — it sounds like something you can choose to have or not have. Like a feature flag for physics.

You can’t choose. The internet and all our datacenters are asynchronous networks that may delay or drop messages. Network partitions happen. As Kleppmann notes, “you don’t really have any choice in this matter.”

The question isn’t whether to tolerate partitions. The question is what to do when they occur. Saying your system is “not partition tolerant” is like saying your house is “not gravity tolerant.” Cool story, but gravity doesn’t care.

Why “Pick Two” Is Wrong

The “pick two of three” framing creates several misconceptions. It’s the kind of oversimplification that sounds great on a whiteboard and falls apart the moment you try to apply it to a real system.

Misconception 1: You Choose Once, Forever

Reality: The choice between C and A happens many times within the same system, at very fine granularity. Different subsystems can make different choices. The choice can vary by operation, by data, even by user.

As Brewer wrote in his 2012 update: “Not only can subsystems make different choices, but the choice can change according to the operation or even the specific data or user involved.”

Misconception 2: Partitions Are Common

Reality: Partitions are rare. Most of the time, your system isn’t partitioned. During normal operation, there’s no reason to forfeit either C or A.

The CAP theorem only forces a choice during a partition. The rest of the time — which is most of the time — you can have both consistency and availability. It’s like an insurance policy: you hope you never need it, but you’d better have a plan for when you do.

Misconception 3: CA Is a Valid Choice

Some people describe their systems as “CA” — consistent and available, but not partition-tolerant. This is confused thinking.

You can’t choose to not have partitions. They happen whether you want them or not. If you’ve designed for CA and a partition occurs, your system will behave unpredictably — probably losing both C and A in the process.

What “CA” usually means in practice: “We designed for a single datacenter and assumed the network is reliable.” That’s not a choice; that’s an assumption that will eventually be violated. It’s the distributed systems equivalent of “it’ll be fine.”

Misconception 4: Systems Are Either CP or AP

Kleppmann makes a compelling argument that most real systems are neither:

“Take any replicated database with a single leader… if a client is partitioned from the leader, it cannot write to the database. Even though it may be able to read from a follower, the fact that it cannot write means any single-leader setup is not CAP-available.”

But if you allow reads from followers with asynchronous replication, those reads won’t be linearizable — so it’s not CAP-consistent either.

Most systems are just “P” — they tolerate partitions but provide neither CAP-consistency nor CAP-availability. They provide something else: useful guarantees that don’t fit neatly into the CAP framework. Which is awkward for everyone who just labeled their architecture diagram.

What Actually Happens During a Partition

Let me make this concrete. Imagine you have database replicas in two datacenters, connected by a network link.

When the network link breaks, you have two choices:

Choice 1: Prioritize Availability Both datacenters continue accepting writes. Clients see no downtime. But the two replicas will diverge — writes in DC1 won’t appear in DC2 and vice versa. You’ve lost linearizability. You’ve gained the ability to sleep through the night. Maybe.

Choice 2: Prioritize Consistency Designate one datacenter as the leader. Only the leader accepts writes. The other datacenter stops accepting writes (and possibly reads) until the partition heals. You’ve maintained linearizability but lost CAP-availability. Your pager, however, is very available.

Neither choice is universally better. It depends on your application.

The Modern Understanding: It’s About Latency

Here’s the insight that changed how I think about CAP: partitions and latency are deeply related.

Brewer’s 2012 update explains: “Operationally, the essence of CAP takes place during a timeout, a period when the program must make a fundamental decision — the partition decision: cancel the operation and thus decrease availability, or proceed with the operation and thus risk inconsistency.”

A partition isn’t a binary thing. It’s a timeout. When a node can’t reach another node within some time bound, it must decide: wait longer (risking availability) or proceed without confirmation (risking consistency).

This means:

• Systems with tighter latency requirements will “see” partitions more often
• What looks like a partition might just be a slow network
• The C vs A trade-off happens constantly, not just during rare network failures

Real-World Strategies I’ve Seen Work

Watching architects handle these trade-offs, I’ve noticed patterns in successful approaches. The good news: none of them involve labeling your system “CP” or “AP” and calling it a day.

Strategy 1: Partition Mode

The best systems don’t just “pick CP or AP.” They explicitly detect partitions and enter a special mode with different rules.

Brewer recommends a three-step approach:

1. Detect the partition
2. Enter partition mode (limit some operations, track history)
3. Recover when communication resumes (restore consistency, compensate for mistakes)

This is more sophisticated than “pick two” — it’s about managing the partition explicitly.

Strategy 2: Operation-Level Decisions

Different operations can make different trade-offs. Consider an e-commerce system:

The system isn’t “CP” or “AP” — it makes different choices for different operations based on their risk profiles.

Strategy 3: Compensation Over Prevention

Sometimes it’s easier to fix inconsistencies after the fact than to prevent them.

Amazon’s Dynamo-powered shopping cart is a famous example. During a partition, both sides accept additions to the cart. After the partition heals, the system merges the carts by taking the union. The consequence: deleted items might reappear. But as the Dynamo paper notes, that’s a minor annoyance compared to losing availability during peak shopping hours — a lost add-to-cart is a lost sale.

This approach requires:

• Understanding which invariants can be temporarily violated
• Having a clear recovery strategy for each violation
• Accepting that some inconsistencies may be visible to users

Strategy 4: CRDTs for Automatic Convergence

Commutative Replicated Data Types (CRDTs) are data structures designed to converge automatically after a partition. They’re a formalization of the “merge by union” approach.

The key insight: if all operations during a partition are commutative (order doesn’t matter), or monotonically increasing on a lattice, the system can automatically merge state without conflicts.

This is an active area of research with practical applications in collaborative editing (Figma), distributed databases (Riak), and real-time collaboration tools.

The Overengineering Boundary

Here’s where CAP-related overengineering gets real. I’ve seen teams go off the deep end in both directions:

Over-applying CAP:

• Building a custom distributed consensus protocol for an application that runs in a single datacenter
• Implementing CRDTs for data that could just use a simple last-write-wins strategy
• Spending weeks debating CP vs AP for a system that serves 50 requests per minute from a single region
• Adding eventual consistency patterns to a system that could just use a single PostgreSQL instance with a read replica

Under-applying CAP (the other kind of overengineering — overengineering your confidence):

• Assuming your single-leader database is “CA” and having no plan for partitions
• Ignoring consistency requirements because “we’re AP” without understanding what that actually means
• Using “eventual consistency” as a hand-wave to avoid thinking about data correctness

You’ve crossed the overengineering boundary when your CAP analysis is more complex than the system it describes. If your “partition handling strategy” document is longer than your application code, something has gone wrong.

The Exception

When IS the deep dive into CAP trade-offs actually warranted?

• Multi-region deployments where network partitions between regions are a realistic and somewhat regular occurrence
• Financial systems where the cost of inconsistency is measured in dollars, lawsuits, or regulatory fines
• Collaborative real-time applications (think Google Docs, Figma) where concurrent edits from partitioned users must eventually converge
• Systems with strict SLAs where you need to formally reason about behavior during failures
• Large-scale distributed databases where you’re actually choosing between consistency models (strong, causal, eventual) and the choice has measurable impact

In these cases, understanding CAP deeply — including its limitations — is essential engineering, not over-engineering. The theorem may be an oversimplification, but the trade-offs it points to are very real.

What to Actually Think About

Instead of asking “Is this system CP or AP?”, ask these questions:

1. What happens to in-flight requests during a partition? Do they timeout? Retry? Return stale data? Return an error? The answer matters more than a CP/AP label.

2. What consistency guarantees do you actually need? Linearizability is expensive. Do you need it? Or would causal consistency, eventual consistency, or read-your-writes consistency be sufficient?

3. What’s your latency budget? Tighter latency requirements mean more “partitions” (timeouts). If you need sub-100ms responses, you can’t wait for cross-datacenter consensus.

4. Which operations have external side effects? Charging a credit card, sending an email, triggering a webhook — these can’t be easily undone. They need different treatment than internal state changes.

5. How will you detect and recover from inconsistencies? If you choose availability during partitions, you’ll have inconsistencies to fix. What’s your strategy?

The Uncomfortable Truth

Here’s what I’ve learned watching architects work through these decisions: there’s no formula. The CAP theorem tells you that you can’t have everything, but it doesn’t tell you what to choose.

The right trade-offs depend on:

• Your specific use case
• Your users’ tolerance for stale data vs. errors
• Your ability to detect and fix inconsistencies
• Your latency requirements
• Your operational maturity

A banking system and a social media feed have very different requirements. Applying the same “CP” or “AP” label to both obscures more than it reveals.

As Kleppmann argues, we should “retire all references to the CAP theorem” and “use more precise terminology to reason about our trade-offs.” I’m not sure I’d go that far — CAP is useful as a starting point for thinking about distributed systems. But it’s only a starting point. And if it’s your ending point, you’ve got a problem.

The real work is understanding your specific requirements and designing a system that makes appropriate trade-offs for each operation, each piece of data, and each failure mode. That’s harder than labeling things “CP” or “AP.” It’s also more useful.

Key Takeaways

• CAP’s definitions are narrow. Consistency means linearizability (not ACID consistency). Availability means every non-failing node responds (not just “the system is up”). Partition tolerance isn’t optional — it’s physics.
• “Pick two” is misleading. The choice between C and A only matters during partitions, which are rare. Most systems aren’t purely CP or AP — they make different trade-offs for different operations.
• Partitions and latency are related. Every timeout is a mini-partition decision. Systems with tight latency requirements face this trade-off constantly.
• Explicit partition handling beats implicit trade-offs. The best systems detect partitions, enter a special mode, and have clear recovery strategies.
• Ask better questions. Instead of “CP or AP?”, ask about specific failure modes, consistency requirements, latency budgets, and recovery strategies. It’s less catchy, but infinitely more useful.

References

1. Brewer, E. “CAP Twelve Years Later: How the ‘Rules’ Have Changed.” IEEE Computer, vol. 45, no. 2, pp. 23–29, February 2012. https://www.infoq.com/articles/cap-twelve-years-later-how-the-rules-have-changed/
2. Gilbert, S. and Lynch, N. “Brewer’s Conjecture and the Feasibility of Consistent, Available, Partition-Tolerant Web Services.” ACM SIGACT News, https://www.comp.nus.edu.sg/~gilbert/pubs/BrewersConjecture-SigAct.pdf
3. Kleppmann, M. “Please stop calling databases CP or AP.” *martin.kleppmann.com*, May 2015. https://martin.kleppmann.com/2015/05/11/please-stop-calling-databases-cp-or-ap.html
4. Kleppmann, M. Designing Data-Intensive Applications. O’Reilly Media, 2017.
5. Shapiro, M., Preguiça, N., Baquero, C., and Zawirski, M. “Conflict-free Replicated Data Types.” SSS’11: Proceedings of the 13th International Conference on Stabilization, Safety, and Security of Distributed Systems
6. Herlihy, M. and Wing, J. “Linearizability: A Correctness Condition for Concurrent Objects.” ACM Transactions on Programming Languages and Systems, vol. 12, no. 3
7. DeCandia, G. et al. “Dynamo: Amazon’s Highly Available Key-value Store.” SOSP ’07: Proceedings of the 21st ACM Symposium on Operating Systems Principles https://www.allthingsdistributed.com/files/amazon-dynamo-sosp2007.pdf

Until next time, keep it simple. Or don’t. We’ll write about it either way. — Overengineering

— The Architect’s Notebook

I’m a Software Engineer learning architecture by watching architects work. If these field notes help you understand architecture better, consider following for more observations every Week.

What’s your CAP confession? Have you ever confidently labeled a system “CP” or “AP” without fully understanding the definitions? Have you built an elaborate partition-handling strategy for a system that runs on a single server? We’ve all been there. Share your story — the best insights come from the most honest mistakes.