Memory Deep Dive: From Registers to RAM — and Why NumPy is Blazing Fast

Hi Everyone, continuing from my last blog where I went deep into how a CPU works, today I wanted to go one level deeper into something that directly connects to why tools like NumPy and PyTorch are so fast. Today I am writing about memory — the different types, how cache works, and what I found when I actually ran experiments on my machine.

The Memory Hierarchy — Fastest to Slowest

Before anything else, let me lay out the full picture. A computer has different types of memory and they are arranged like a hierarchy from fastest to slowest:

Registers → L1 Cache → L2 Cache → L3 Cache → RAM (DRAM) → SSD → HDD

The closer the memory is to the CPU, the faster it is — and the smaller it is. Registers are the fastest but they just hold the values the CPU is operating on at that exact instant. Cache is the next layer the CPU looks into immediately after registers.

How Cache Actually Works — Spatial and Temporal Locality

Cache is a small, very fast memory located on or very close to the CPU. When the CPU needs data, it first checks the L1 cache. If the data is not found (a cache miss), it checks the L2 cache, then the L3 cache, and finally RAM. If the data is not in RAM, it may need to be fetched from storage (SSD/HDD).

The CPU does not fetch just one value or one bit at a time from RAM. That would be too slow. Instead it fetches in chunks called a cache line, and each cache line is 64 bytes at a time.

The cache is smart about what it predicts you will need next. It uses two strategies:

Spatial Locality — If you are looping through a list of numbers, the cache predicts that the next few numbers in memory are probably needed soon. So it loads them in advance. This is why sequential access is fast.

Temporal Locality — If you are using the same variable again and again in a loop, the cache keeps it nearby because it knows you will need it repeatedly.

Memory Flow

Inside RAM — How Big Is It Really?

Let me put some numbers to this. My machine has 16GB of RAM.

First let’s understand what 16GB actually means in bits:

16 GB = 16 × 1024 × 1024 × 1024 bytes = 17,179,869,184 bytes — that is approximately 17.18 billion bytes.

Since each byte is 8 bits, the total bits in 16GB = 17.18 billion × 8 = approximately 137 billion bits.

Since each memory address in RAM holds exactly 1 byte (8 bits), 16GB of RAM gives us exactly 17,179,869,184 individual memory addresses — roughly 17.18 billion addresses.

Even though RAM can hold this much data, a 64-bit CPU can only work on 64 bits — that is 8 bytes — per clock tick through its 64 data wires at once.

A 64-bit CPU can theoretically address up to ²⁶⁴ = 18.4 quintillion unique memory addresses. That is an astronomically large number compared to the 17.18 billion addresses in 16GB of RAM — which is exactly why 64-bit systems can support far more than 16GB of RAM if needed.

Why Python is Heavy on Memory — The x = 2 Example

Now here is something that surprised me when I dug into it. If I write x = 2 in Python, it does not just store the number 2. Python wraps it in a full object in RAM like this:

• Reference count (8 bytes) — tracks how many variables are pointing to this object
• Type pointer (8 bytes) — points to the type definition, telling Python this is an integer
• Actual value (4 bytes) — the number 2 itself
• Padding and flags — to align everything properly

Total: 28 bytes just to store the number 2. That feels wasteful, and it is — but it is what makes Python flexible and easy to use.

The Experiment — Sequential vs Random Access

To understand how cache works in practice I ran an experiment with a Python list of 10 lakh numbers (1 million numbers).

In a normal Python list, each item is stored as a pointer — it does not directly hold the number. Instead it holds an address that points to another address where the actual Python object is stored with the real value. So for every number the CPU needs to:

Fetch the pointer → Follow the pointer → Fetch the actual integer object

The cache line is 64 bytes. Since each pointer is 8 bytes, one cache line loads 8 pointers at a time. But those 8 pointers each point somewhere else in memory — so the CPU still has to make 8 more fetches to get the actual values.

Result when accessing sequentially: Sequential Python list: 2.745 seconds

Now I shuffled the indices randomly so the loop jumps around in memory with no pattern. The cache cannot predict spatial locality anymore. Every fetch is a cache miss — meaning the CPU has to go all the way back to RAM each time.

Result with random access: Random Python list: 10.119 seconds

Nearly 4x slower — just because the access pattern changed.

NumPy — Why It Is So Much Faster

Now I ran the same experiment with a NumPy array of 10 lakh numbers.

NumPy stores values completely differently. Instead of pointers to Python objects, NumPy stores the actual values directly in contiguous memory — one after another, each taking exactly 8 bytes. No pointers. No reference counts. No type wrappers.

This means when the cache line fetches 64 bytes, it gets 8 actual values ready to compute — not 8 pointers that still need to be chased.

Result: Sequential NumPy: 0.008 seconds

That is over 300 times faster than the Python list.

Even with random shuffling, NumPy stays fast — and here is the biggest reason why. When Python runs a loop, every iteration goes through the Python interpreter — checking types, managing objects, handling overhead. NumPy bypasses all of that because under the hood it runs compiled C code like this:

for(i = 0; i < n; i++) { sum += data[i]; }

No interpreter. No Python objects. No overhead. Just raw computation directly on memory.

What This Means for AI and ML

This is exactly why frameworks like PyTorch and TensorFlow are built on top of NumPy-style contiguous memory storage. When you are multiplying matrices with millions of numbers, the difference between chasing pointers and reading direct values is the difference between waiting minutes and getting results in seconds.

Understanding this made me realise that learning AI is not just about knowing which function to call. Knowing why something is fast under the hood helps you write better code and make better decisions.

That is what I learned today. If I have made any mistakes please correct me in the comments — I am learning in public and every correction helps. See you in the next one.