← Back to blogs

The Boundaries of Physics: CAP and the PACELC You Actually Live With

CAP tells you what happens during a partition. PACELC tells you what happens the other 99.9% of the time: you trade latency for consistency on every single write, partition or not. If you only design for CAP, you have ignored the trade-off that shapes your p99.

1. CAP, Precisely Stated

In the presence of a network partition (P), a system can provide at most one of: Consistency (C — linearizability) or Availability (A — every non-failed node responds). The theorem is narrower than folklore: it says nothing about normal operation, it compares linearizability against total availability, and it assumes the partition is real. Many "CA systems" in slideware are CA only because the slides omit P.

• CP: Refuse to answer on the minority side. etcd, Consul, Spanner, CockroachDB (majority required).
• AP: Answer from any replica, reconcile later. Cassandra, DynamoDB, CouchDB (tunable, but AP by default).

In production the partition is not hypothetical — it is a rolling kernel upgrade that holds a rack's switch for 11 seconds, an AZ failure, or a noisy neighbor saturating the NIC. Your "rare partition" happens monthly.

2. PACELC: The Trade-Off You Pay Every Day

Daniel Abadi's PACELC completes the picture: If Partition, trade Availability vs. Consistency; Else, trade Latency vs. Consistency.

Even when the network is healthy, a strongly consistent write must contact a quorum before acknowledging. That costs a round trip to the farthest quorum member — typically cross-AZ. An eventually consistent write can ack after one replica and gossip the rest asynchronously. The gap is your p50/p99 write latency and, at scale, your throughput ceiling.

PA/EL system (e.g., Cassandra ONE):  P → Available, E → Low Latency (ack 1 replica)
PC/EC system (e.g., CockroachDB):    P → Consistent,  E → Consistent (quorum ack)
PC/EL system (e.g., Spanner w/ TrueTime): P → Consistent, E → Low latency via clock uncertainty
-- No PA/EC system exists that is both available during partitions and consistent otherwise
-- without paying latency — that would be CA, which requires no partitions ever.

Select a database below. The matrix highlights its position on both axes — what it does when partitioned (left/right) and what it does when healthy (top/bottom). The point is not to rank systems, but to make the trade-off you already bought visible.

3. Reading Real Systems on the Matrix

Defaults are not destiny — every system below exposes knobs that move it across quadrants, usually at 2 a.m. when you change a consistency knob to stop an outage:

• Cassandra / DynamoDB (PA/EL by default): ONE / eventual reads are single-replica ack. Dial to QUORUM and you pay cross-AZ quorum latency and risk unavailability on the minority side — you moved to PC/EC without changing binaries.
• CockroachDB / etcd (PC/EC): Every write goes through Raft quorum. You get linearizability, but your p99 is the p99 of the quorum round trip. No background reconciliation, but higher tail latency is structural.
• Spanner (PC/EL variant): Uses TrueTime uncertainty to shave the "else" latency: it waits out the clock uncertainty window (~7 ms) instead of a full quorum cross-region round trip for read-only transactions. Still CP during partitions — just cheaper when healthy.
• Cosmos DB: Makes the trade-off explicit with five named levels (strong → eventual), each a different point on the PACELC spectrum. The lesson: naming the trade-off lets teams choose per workload.

4. Choosing Before You Are Forced To

Decision framework used in production reviews:

1. Name the invariant. "Cart total must not oversell" → PC/EC. "Like count may be stale 5s" → PA/EL.
2. Budget the latency. If SLO is p99 < 30 ms cross-AZ, PC/EC may violate it on every quorum write.
3. Plan the partition drill. What does the minority do? Refuse, serve stale, or fork? Document and test it.
4. Expose the knob. Make consistency tunable per request or per table, not per cluster — workloads differ.

-- CockroachDB: per-txn choice
SET TRANSACTION AS OF SYSTEM TIME '-5s';  -- stale read → EL
-- Cassandra: per-query choice
SELECT * FROM orders USING CONSISTENCY QUORUM; -- EC
SELECT * FROM orders USING CONSISTENCY ONE;    -- EL
-- The quadrant is a query-level decision, not just a vendor label.

5. The Limit of Trade-Offs

CAP and PACELC tell you what you must give up. They do not tell you how to achieve what you decided to keep. A PC/EC choice still needs a protocol that guarantees linearizable agreement on the log across crashes and partitions — with the failure model you already chose and clocks you already distrust.

Knowing what must be sacrificed still leaves how to build what you decided to keep — a protocol that keeps a single log linearizable while leaders change and messages disappear. That how is consensus, and it is where the primitives from this series finally come together. The next series picks up there with Raft and Paxos, operation by operation.