Behind the Code: Engineer’s Guide to Python Internals and Memory Optimization (Part-1)

If you run Python services at scale, you’ve likely noticed a frustrating trend: your apps consume significantly more memory and CPU than equivalent services written in Go, Rust, or Java.

When your infrastructure costs start scaling faster than your user base, standard optimization advice like “just use a list comprehension” won’t cut it. To truly optimize Python, you have to look past its clean syntax and understand its runtime engine: CPython.

As software engineers, understanding what happens under the hood changes how we write production code. Let’s dive deep, explore how core data structures are laid out in memory, and analyze the engineering decisions that impact system performance.

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

Given the technical depth of this topic, this series is divided into three parts:

• 📌 Part 1: The High-Level Productivity vs. Infrastructure Cost Trade-off & Scalar Object Costs
• ⏰ Part 2: Structures Deep Dive — Lists, Sets, and Dictionaries Under the Hood (Coming Soon)
• ⏰ Part 3: Complete Picture — Putting Python Internals Into Production Practice (Coming Soon)

The High-Level Productivity vs. Infrastructure Cost Trade-off

Python abstracts away memory management, pointer arithmetic, and explicit type declarations. While this accelerates development velocity, it shifts the computational burden directly to the runtime environment.

In compiled languages like Go or Rust, a variable often maps directly to a specific memory address containing a raw binary value. In CPython, variables are names bound to object references. On a typical 64-bit build, those references are generally 8 bytes wide, but the exact representation is implementation-dependent. This structural overhead can cause infrastructure costs to scale non-linearly under heavy data-processing workloads, prompting some organizations to rewrite microservices in statically typed languages once they hit massive scale.

However, a deep understanding of CPython internals allows you to write highly optimized Python that minimizes these costs without abandoning the ecosystem.

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

Python Object Memory Representation: PyObject

In CPython, everything is an object, and every object lives on the heap. Even a simple integer is not just 64 bits of data; it is a complex C structure. At the core of every Python object is the PyObject structure (or PyVarObject for variable-length structures like lists and strings).

┌──────────────────────────────────────────────────┐
│  PyObject Header (16 bytes on 64-bit system)     │
├──────────────────────────────────────────────────┤
│  ob_refcnt (8 bytes)  - Reference count          │
│  ob_type (8 bytes)    - Pointer to type object   │
├──────────────────────────────────────────────────┤
│  Object-specific data (variable size)            │
│  - For list: array of PyObject* pointers         │
│  - For dict: hash table (indexes + dense table)  │
│  - For int: digit array (arbitrary precision)    │
│  - For str: compact UTF-8/UTF-16/UTF-32 data     │
└──────────────────────────────────────────────────┘

1. The Header Details

• **ob_refcnt (8 bytes / 64 bits):** This field tracks how many variables, containers, or internal frames reference this specific memory address. It uses a standard 64-bit signed/unsigned integer (Py_ssize_t).
• **ob_type (8 bytes / 64 bits):* This is a raw C pointer (`struct _typeobject ) pointing to the type object (likePyLong_Type,PyList_Type`, or a custom class). This pointer is the reason Python doesn't need variable type declarations; the object itself carries its type identity.

2. Variable-Length Header (PyVarObject) Extension

For containers and variable-length structures (like list, dict, str, bytes), CPython uses a slightly extended header macro called PyVarObject. It adds exactly one more field right after ob_type:

• **ob_size (8 bytes):** Tracks the number of items in a list, characters in a string, or elements currently allocated.

3. Object-Specific Data Payloads

• For list (PyListObject): It holds an ob_size field in the header, and the payload is a PyObject ob_item pointer. This pointer references a contiguous array of other 8-byte pointers, which in turn point to the actual items on the heap.
• For dict (PyDictObject): It points to a split-table architecture introduced in Python 3.6. It contains an indices array (sparse array of bytes/ints) and a tightly packed, sequential entries array containing the actual hashes, key pointers, and value pointers.
• For int (PyLongObject): Python integers have arbitrary precision. The payload is an array of digit values (typically 30-bit integers stored in 32-bit fields). If the number is small, it uses 1 digit; if it's massive, it allocates more digits dynamically.
• For str (PyASCIIObject / PyCompactUnicodeObject): Modern Python uses a flexible string representation (PEP 393). Depending on the highest character code point in the string, the payload shifts dynamically between 1 byte (ASCII/Latin-1), 2 bytes (UCS-2/UTF-16), or 4 bytes (UCS-4/UTF-32) per character, stored inline right after the header to maximize CPU cache friendliness.

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

The True Memory Blueprint of Python Scalar Types

1. Integers (PyLongObject)

Python integers have arbitrary precision (they never overflow; they automatically grow to accommodate massive numbers). Because of this design choice, they cannot be stored as simple 64-bit values.

Real Cost Breakdowns

• Small Integer (0 or 1): 28 bytes
• Large Integer (2^30 and up): 32 bytes (allocates an extra 4-byte “digit” segment)

import sys
print(sys.getsizeof(0))           # 28 bytes
print(sys.getsizeof(1))           # 28 bytes
print(sys.getsizeof(1073741824))  # 32 bytes (Requires two 30-bit digits)

CPython Optimization Note: To offset this 28-byte tax on basic operations, CPython pre-allocates a static global array of integers from **-5 to 256** at startup. Referencing these values simply reuses an existing memory pointer instead of performing a fresh heap allocation.

2. Floats (PyFloatObject)

Unlike integers, Python floats do not have arbitrary precision. They map directly to standard IEEE 754 double-precision variables (64-bit C doubles). However, they are still wrapped in the standard PyObject structure.

Real Cost Breakdown

• Any Float: 24 bytes (16 bytes header + 8 bytes raw double value)

import sys
print(sys.getsizeof(3.14))  # 24 bytes

3. Strings (PyASCIIObject / PyCompactUnicodeObject)

Python strings underwent a massive rewrite in PEP 393 to optimize memory. Instead of storing everything uniformly as UTF-8 or UTF-32, CPython adjusts the underlying byte-width based on the lexical content of the string.

Real Cost Breakdown

• Empty String: 49 bytes (Base metadata header size)
• ASCII String: 49 bytes + 1 byte per character
• Latin-1 / Extended ASCII: 73 bytes + 1 byte per character
• UCS-2 (Emojis / Cyrillic / Kanji): 74 bytes + 2 bytes per character
• UCS-4 (Rare scripts / complex symbols): 76 bytes + 4 bytes per character

import sys
print(sys.getsizeof(""))         # 49 bytes (Header only)
print(sys.getsizeof("hello"))    # 54 bytes (49 + 5 bytes ASCII)
print(sys.getsizeof("🚀"))       # 80 bytes (Switches system to a wider character layout)

Structural Overhead in Primitive Collections

1. Lists (PyListObject)

A Python list does not contain direct data values. It is a contiguous array of 8-byte pointers pointing to objects elsewhere on the heap. Furthermore, lists use an over-allocation strategy to make appending O(1) amortized.

Real Cost Breakdown

• Empty List: 56 bytes (Header + allocation trackers)
• Populated List: 56 bytes + (8 bytes * Allocated Slots)

When you create a list, CPython allocates more memory slots than you currently need. If you have 5 items, Python might allocate 8 structural slots.

import sys
my_list = []
print(sys.getsizeof(my_list))  # 56 bytes
my_list.append(1)
print(sys.getsizeof(my_list))  # 88 bytes (Allocated slots jumped from 0 to 4: 56 + 4 * 8 = 88)

The Invisible Cost Example

If you create a list containing 1,000 integers:

• The list structural object itself: 56 + (1000 * 8) = 8,056 bytes
• The 1,000 distinct integer objects on the heap: 1000 * 28 = 28,000 bytes
• Total real cost: ~36,056 bytes (A native C or Go array of 1,000 integers takes exactly 8,000 bytes).

2. Tuples (PyTupleObject)

Tuples are fixed-length, immutable sequences. Because they cannot be resized, they do not require over-allocation tracking headroom.

Real Cost Breakdown

• Empty Tuple: 40 bytes
• Populated Tuple: 40 bytes + (8 bytes * Length)

import sys
print(sys.getsizeof(()))        # 40 bytes
print(sys.getsizeof((1, 2, 3))) # 64 bytes (40 + 3 * 8 bytes)

Performance Win: Tuples combine their metadata header and their payload pointers into one contiguous memory allocation block. This makes them faster to instantiate and highly friendly to the CPU cache compared to lists.

3. Dictionaries (PyDictObject)

Dictionaries are highly advanced hash tables. To maintain insertion order and eliminate structural layout gaps, modern Python uses a Split Table Architecture (a tight byte-index array pointing to a dense table of 24-byte payload slots).

Real Cost Breakdown

• Empty Dictionary: 64 bytes (Internal base structure)
• First Insertion: Minimum allocation leaps straight to 248 bytes. CPython forces a minimum initial partition capacity of 8 hash slots to mitigate early collisions.

import sys
my_dict = {}
print(sys.getsizeof(my_dict))  # 64 bytes
my_dict["key"] = "value"
print(sys.getsizeof(my_dict))  # 248 bytes (Minimum hash pool initialized)

Production Optimization Playbook

By default, every custom class instance you build in Python generates an internal dictionary (__dict__) to allow arbitrary creation of properties on the fly. This means even an object with just two variables carries massive memory baggage.

The Expensive Way

import sys

class Coordinates:
    def __init__(self, x, y):
        self.x = x
        self.y = y

p = Coordinates(1, 2)

# 1. Base struct size
base_size = sys.getsizeof(p)                      # 48 bytes
# 2. Hidden dictionary size
dict_size = sys.getsizeof(p.__dict__)             # 104 bytes (varies by Python version, up to 216B)

print(f"Total Standard Cost: {base_size + dict_size} bytes") 
# Output: Total Standard Cost: 152 bytes

The Optimized Production Way

• Enforce __slots__ on Data Models: When designing stateful classes or data transfer objects (DTOs) that will scale to millions of concurrent instances, explicitly define __slots__ to prevent dictionary overhead.

class OptimizedCoordinates:
    __slots__ = ('x', 'y')
    def __init__(self, x, y):
        self.x = x
        self.y = y

p_opt = OptimizedCoordinates(1, 2)

base_opt_size = sys.getsizeof(p_opt)              # 48 bytes

# Trying to access p_opt.__dict__ will raise an AttributeError because it doesn't exist!
print(f"Total Optimized Cost: {base_opt_size} bytes")
# Output: Total Optimized Cost: 48 bytes

• Pre-size Containers Safely: If you know you need to ingest a large number of items into a list, appending sequentially will cause multiple memory reallocations and copies. While Python doesn’t have an explicit capacity initializer like Go (make([]T, 0, cap)), you can pre-allocate a list of fixed references if the size is known:

# Pre-allocated allocation block 
data_store = [None] * 1_000_000

• Use Generator Expressions for Heavy Pipelines: Avoid generating intermediate lists when filtering or transforming large datasets:

# Memory Intensive: Allocates an intermediate list of 10 million integers 
total = sum([x * 2 for x in range(10_000_000)])  

# Stream Optimized: Evaluates items one at a time using a generator 
total = sum(x * 2 for x in range(10_000_000))

• Leverage Built-ins Correctly: For high-performance membership checks, prefer sets over lists. Checking if item in collections_list performs a linear O(n) scan across the pointer array. Converting that collection to a set optimizes the check to a fast O(1) hash lookup.

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

📚 References

• Introduction to Data Structures in Python (Visually Explained) — Data with Baraa [00:13]
• CPython Data Model Official Documentation

To be Continued… ✍️

In Part 2, we will perform a deep architectural dive into how Python lists grow, the mechanics of open-address probing in dictionaries, and how hash collisions impact live infrastructure performance. Stay tuned!