When Parallelism Actually Speeds Things Up (Amdahl’s Law)

Part 5 of the “Concurrency & Parallelism Explained” series. You threw 8 cores at the problem and got 2x, not 8x. Math predicted that exactly.

A slow job is taking too long, so you parallelize it across 8 cores, expecting it to finish 8 times faster. It finishes about 2 times faster. You add more cores; it barely improves. Disappointed, you conclude parallelism is overrated. But parallelism didn’t fail you — unrealistic expectations did, and there’s a precise law that predicts exactly how much speedup you’ll actually get. It’s called Amdahl’s Law, and understanding it is the difference between parallelizing the right things and wasting effort on the wrong ones.

Let me show why more cores so often disappoint, and how to know in advance whether parallelism will actually help your specific problem.

Adding cores only speeds up the part of your program that can run in parallel. The part that must run sequentially doesn’t care how many cores you have — and that sequential part quietly sets your ceiling.

First: Parallelism ≠ Concurrency

A quick but crucial distinction this series has been building toward:

• Concurrency — dealing with many things at once (structure). The event loop (Part 4) is concurrent on one core.
• Parallelism — doing many things at once (execution). Requires multiple cores genuinely running simultaneously.

This article is about parallelism — true simultaneous execution — and when it pays off. (You can have concurrency without parallelism, and vice versa.)

Amdahl’s Law: The Ceiling You Can’t Beat

Here’s the uncomfortable math. If some fraction of your program must run sequentially (it can’t be parallelized), that fraction caps your maximum speedup no matter how many cores you add:

If 20% of the work is inherently sequential (only 80% parallelizable):
  with infinite cores, max speedup = 1 / 0.20 = 5x   ← that's the CEILING

  8 cores:  speedup ≈ 1 / (0.2 + 0.8/8)  ≈ 3.3x   (not 8x!)
  16 cores: speedup ≈ 1 / (0.2 + 0.8/16) ≈ 4.0x
  ∞ cores:  speedup → 5x and never more

That’s why 8 cores gave you ~2–3x, not 8x: a chunk of your job — reading the file, the final aggregation, the coordination — runs sequentially, and that chunk is the ceiling. The sequential fraction, not the core count, determines your limit. Doubling cores past a point buys almost nothing.

Fix #1: Parallelize the Right Workloads

Parallelism helps when work is CPU-bound and divisible into independent pieces that barely need to coordinate. The ideal case is “embarrassingly parallel” — chunks that don’t talk to each other:

GREAT for parallelism (independent, CPU-heavy):
  + resizing 10,000 images
  + processing each row of a huge dataset independently
  + running a simulation across many parameter sets
  + brute-force search over a partitioned space

POOR for parallelism:
  - inherently sequential steps (each needs the previous result)
  - tasks that constantly share/synchronize state (locking eats the gains)
  - I/O-bound work (use async, not cores — see Part 4)

Before parallelizing, ask: can this work be split into pieces that run independently? If the pieces constantly wait on each other or on shared locks, parallelism won’t deliver — the coordination overhead devours the speedup.

Fix #2: Account for Coordination Overhead

Amdahl’s Law is optimistic — it ignores the cost of parallelizing, which is real:

• Splitting and merging the work takes time (the sequential part grows).
• Communication between workers (especially across processes) costs serialization and IPC.
• Synchronization (locks, barriers) makes workers wait on each other.
• Spawning threads/processes has startup cost.

For small workloads, this overhead can make the parallel version slower than the sequential one. Parallelism pays off when each chunk of real work is large enough to dwarf the coordination cost. Don’t parallelize a job that takes 10ms — you’ll spend more than 10ms just coordinating.

Fix #3: Measure, Don’t Assume

Like all performance work (the profiling lesson), parallelism should be measured, not guessed:

# Measure sequential first — your baseline and your Amdahl sequential fraction
t0 = time.perf_counter(); sequential_version(); print(time.perf_counter() - t0)

# Then parallel, and compute ACTUAL speedup
from concurrent.futures import ProcessPoolExecutor
t0 = time.perf_counter()
with ProcessPoolExecutor() as pool:
    list(pool.map(work, chunks))
print(time.perf_counter() - t0)   # divide into baseline → real speedup

Measure the real speedup against more cores. If going from 4 to 8 cores barely moves the needle, you’ve hit your Amdahl ceiling — adding hardware is wasted money. The measurement tells you when to stop.

Putting It Together

1. Parallelism = true simultaneous execution on multiple cores (≠ concurrency).
2. Amdahl’s Law — the sequential fraction caps your speedup; more cores hit diminishing returns.
3. Parallelize independent, CPU-bound, large-enough chunks — not sequential, chatty, or I/O work.
4. Subtract coordination overhead and measure real speedup; stop when cores stop helping.

The Takeaway

Parallelism disappoints people because they expect linear speedup — 8 cores, 8x faster — when Amdahl’s Law guarantees the opposite: whatever fraction of your work is stuck running sequentially sets a hard ceiling that no amount of hardware can break. The skill isn’t throwing cores at problems; it’s recognizing which problems are actually parallelizable (independent, CPU-bound, coarse-grained) and measuring the real speedup instead of assuming it. Parallelize the embarrassingly-parallel, leave the sequential and I/O-bound to other tools, account for coordination cost, and stop adding cores when the math says they’ve stopped helping. Used that way, parallelism delivers — just never as much as the naive dream, and exactly as much as the law predicts.

That’s the whole series in one arc: understand shared memory (threads vs processes), protect it from races, avoid deadlocks, exploit waiting with the event loop, and exploit cores with parallelism — each tool matched to the problem it actually solves.

This is Part 5 — the finale of the Concurrency & Parallelism Explained series. You’ve now got the set: threads vs processes, race conditions, deadlocks, the event loop, and the real limits of parallelism.

What’s a time parallelism gave you way less speedup than you hoped? 👇