Memory Models & Segmentation Explained: Real Mode vs Protected Mode vs Long Mode

Photo by Maxim Berg on Unsplash

Have you ever asked yourself: “How does my CPU know where memory begins and ends?” Or, better yet — why does that old DOS game crash on your modern machine?

Welcome to the fascinating world of memory models and segmentation — a topic that may sound dry on the surface, but underneath is full of architectural drama, legacy hangovers, and some truly beautiful engineering.

Today, we’re diving deep into three essential modes of memory operation in x86 architecture:

• Real Mode (hello, 1980s!)
• Protected Mode (welcome, multitasking OSes)
• Long Mode (64-bit modernity)

As always, I’ll break it down in a friendly, classroom-style vibe with a few side jokes, practical insights, and just enough technical rigor to make your inner nerd smile.

Why Memory Models Exist (A Simple Analogy)

Imagine you’re trying to organize books in a library.

• In the early days, you had only one shelf: Real Mode. Every book had to fit here.
• Then came bigger libraries with security guards and shelf managers: Protected Mode.
• Finally, you built a multi-story skyscraper with automated elevators and infinite storage: Long Mode.

Each of these “modes” tells the CPU how to interpret memory addresses and how to interact with RAM. Your processor operates in only one mode at a time, and the mode determines what kinds of instructions and memory models it can use.

Let’s meet them.

Real Mode: The Legacy of the 8086

The year is 1978. Intel releases the 8086 processor. It could address exactly 1 MB of memory.

That’s it.

Memory Addressing in Real Mode:

• Memory is accessed via segment:offset pairs.
• A segment register (like CS, DS, ES) defines a 64 KB segment.
• The physical address = segment * 16 + offset.

So 0x1234:0x5678 = 0x12340 + 0x5678 = 0x179B8

This 20-bit addressing scheme gives a total of 1,048,576 bytes = 1 MB of addressable memory.

Limitations of Real Mode:

• No concept of memory protection or privilege levels.
• No multitasking or process isolation.
• No memory paging or virtual memory.

It’s like a classroom with no rules — everyone can access everything. If one student (process) messes up the board (memory), it ruins it for everyone.

Fun Fact:

Even modern Intel CPUs boot in Real Mode for backward compatibility. That’s right — your shiny i9 CPU starts like it’s 1981.

Protected Mode: Paging, Privileges, Power

In the mid-1980s, Intel gave us the 80286 and 80386, introducing Protected Mode — so named because it allowed protection of memory between processes.

Protected Mode is where modern OSes like Windows, Linux, and macOS began to thrive.

Features of Protected Mode:

1. 32-bit memory addressing: Access up to 4 GB of RAM.
2. Segment Descriptors: Instead of hardcoding addresses, segment registers point to descriptor tables (GDT/LDT).
3. Paging: Translate logical addresses to physical addresses via page tables.
4. Privilege Levels (Rings):

• Ring 0: Kernel mode
• Ring 3: User mode

Fault Isolation: A misbehaving process can no longer crash the entire system.

Think of Protected Mode as an office where every employee (process) gets their own cubicle (memory space), ID badge (ring level), and permissions.

Memory Model:

In Protected Mode, segmentation still exists but works differently:

• A segment register (like CS, DS, SS) contains a selector, which indexes into a descriptor table (GDT or LDT).
• The descriptor defines the base address, limit, and access rights.

But most modern OSes configure the segments to cover the full address space and rely on paging for memory management.

Example: Flat Memory Model

Most 32-bit systems set up segments to cover the entire 4GB space with base = 0.

mov eax, [0x12345678]

Just works. No segment fuss. Segmentation becomes nearly invisible.

Long Mode: Welcome to 64-Bit Nirvana

Fast forward to the 2000s. We needed more RAM, more power, and fewer architectural headaches. Enter x86–64 and its 64-bit wonderland: Long Mode.

Features of Long Mode:

• 64-bit addressing (theoretically supports ²⁶⁴ bytes of memory).
• In practice, current CPUs support 48-bit or 52-bit addressing.
• Flat memory model: Segmentation is almost completely disabled.
• Mandatory paging: Long mode requires paging to function.
• Larger general-purpose registers: EAX becomes RAX, and so on.

In Long Mode, your CPU works with flat 64-bit virtual addresses mapped to physical memory via a 4-level page table hierarchy.

Segmentation: Still There… Barely

Here’s the kicker:

• In Real Mode: Segmentation is everything.
• In Protected Mode: Segmentation is used but mostly flattened.
• In Long Mode: Segmentation is essentially ignored (except FS and GS for thread-local storage).

If you’re reversing malware or digging into OS internals, understanding segmentation is still valuable — especially when dealing with 16-bit legacy code or VM introspection.

Bonus: Thread-Local Storage via FS/GS

Even in Long Mode, the FS and GS segment registers are used!

Example:

• In Windows x64, GS points to the Thread Environment Block (TEB).
• In Linux, FS is used for thread-local data.

mov rax, gs:[0x30]  ; Access thread-specific data in Windows

Security Implications

Here’s where memory models matter most to security professionals:

Buffer Overflows & Segment Boundaries

In Real Mode, a buffer overflow can wipe out BIOS data. In Protected Mode, segment descriptors prevent writing outside bounds (in theory). In Long Mode, it’s all about page permissions — DEP, ASLR, and SMEP matter more than segments.

Privilege Escalation

Privilege levels (rings) enforce security. A compromised process in Ring 3 can’t touch Ring 0… unless a bug (like a kernel exploit) breaks the wall.

Legacy Bugs

Some old BIOSes or bootloaders assume Real Mode behavior and crash under Long Mode. If you’re doing reverse engineering, you’ll run into these quirks.

Hands-On Lab Idea (for practice)

Want to play with modes?

1. Boot into Real Mode:

• Use DOSBox or Bochs to simulate 16-bit Real Mode environments.

2. Switch to Protected Mode:

• Try writing a custom bootloader that switches to Protected Mode (search: “OSDev protected mode tutorial”).

3. Explore Long Mode:

• Use GDB and objdump to analyze 64-bit programs.
• Inspect segment registers with:

info registers

cat /proc/self/maps

Summary Table

| Mode          | Year | Bits | Max Memory      | Segmentation | Paging | Used By                              |
|---------------|------|------|-----------------|--------------|--------|--------------------------------------|
| Real Mode     | 1978 | 16   | 1 MB            | Mandatory    | ❌     | BIOS, Bootloaders                     |
| Protected Mode| 1982 | 32   | 4 GB            | Optional     | ✅     | Legacy OSes, 32-bit                   |
| Long Mode     | 2003 | 64   | 2⁶⁴ (virtual)   | Barely Used  | ✅     | Modern OSes                           |

Key Takeaways

• Memory modes define how the CPU addresses memory.
• Real Mode is ancient, Protected Mode is transitional, Long Mode is the modern standard.
• Segmentation mattered… and still does in low-level and security contexts.
• Understanding memory models helps you reverse engineer, build exploits, and write your own kernels (yes, really).

Final Words: Memory Is Not Just RAM

When we talk about memory in security, we’re not just talking about physical RAM sticks. We’re talking about how the CPU views memory, how it protects it, and how clever attackers bypass those protections.

So if you’re serious about low-level security, malware analysis, or OS internals — understand memory models like your life depends on it. Because someday, your rootkit detector might.

Until then, keep segmenting (or unsegmenting), and I’ll see you in the next mode.