Image edited by the author using AI

The AI Memory War Explained: Why HBM, GDDR, and DRAM Are the Next Semiconductor Bottleneck Powering Artificial Intelligence

From SK Hynix’s HBM3E stacks and Micron’s DRAM fabs to the memory soldered onto NVIDIA GPUs, the race for faster, denser chips is quietly deciding how powerful AI models can become — and which companies dominate the future of AI.

I remember the first time I heard a chip engineer say, “The GPU is not the problem. The memory is.” I nodded politely and quietly looked it up afterward. That was a few years ago. Today, that statement has become the defining tension of the entire AI industry — and it’s affecting everything from the price of your gaming PC RAM to whether a $40,000 AI server can actually run the model you’re training.

So let’s slow down and actually understand what’s happening.

First, why does AI need so much memory? Recognizing this helps the industry understand the critical impact of memory shortages on AI progress and investment decisions. Before we talk about memory types, let’s talk about why memory matters at all for AI — because this is the part most tech coverage skips.

When a large language model like GPT-4 or Llama 3 runs — generating text, analyzing images, translating code — it doesn’t work like a human brain pulling on intuition. It loads billions of numerical parameters (called weights) into memory and performs rapid mathematical operations across all of them simultaneously. The bigger the model, the more parameters it has. The more parameters, the more memory you need — and the faster that memory has to deliver data to the processor.

Here’s the thing nobody talks about: the processor is almost always waiting on memory. The chips doing the math are fast enough. The bottleneck is the pipeline feeding them. If memory can’t keep up, you’ve built a Formula 1 car with a garden hose for a fuel line. Engineers call this gap the “memory wall” — and solving it, or at least managing it, is what the entire AI hardware arms race is actually about, beneath all the marketing.

Meet the Three Contenders

Not all memory is created equal. There are really three different types in play right now, each serving a different master, each caught up in this shortage in a different way.

HBM: The One Everyone Is Fighting Over

HBM — High Bandwidth Memory — is the premium, purpose-built fuel that powers AI accelerators. The key difference between it and the RAM in your laptop isn’t just speed — it’s architecture. Instead of chips sitting flat on a circuit board, HBM stacks multiple memory chips vertically, like a tiny skyscraper, and places them right next to the processor on the same package. The connection between them is incredibly short and incredibly wide, which is why the data throughput is so jaw-dropping.

To give you a sense of scale: Nvidia’s H100 — the workhorse of most AI data centers for the past two years — ships with 80 GB of HBM3 delivering 3.35 terabytes of data per second. The H200 pushed that to 141 GB at 4.8 TB/s. And Nvidia’s newest Blackwell B200, announced at GTC in March 2024, goes to 192 GB at 8 TB/s per chip. For reference, a typical DDR5 module in your home PC delivers around 50–80 GB/s. The gap between what your computer uses and what an AI accelerator needs isn’t a rounding error — it’s roughly two orders of magnitude. (All specs confirmed via Nvidia’s official product pages and SK Hynix’s manufacturer press releases.)

The catch? HBM is extraordinarily expensive and brutally difficult to make. This complexity underscores the industry’s resilience and the importance of strategic planning in supply chains.

GDDR: The Workhorse Getting Squeezed

If HBM is the exotic racehorse, GDDR — Graphics Double Data Rate memory — is the reliable quarter horse that powers most of the world’s gaming GPUs. It doesn’t stack chips the way HBM does, but it achieves solid bandwidth through wide interfaces and high data rates, at a fraction of the cost.

The latest generation, GDDR7, is increasingly relevant for AI inference — the process of actually running a trained model rather than training it from scratch. But here’s the wrinkle: according to TrendForce’s December 2025 industry analysis, GDDR7 requires about 1.7 times the manufacturing capacity of standard DRAM per gigabyte produced. It’s not as resource-hungry as HBM, but it’s not cheap to make either. And when manufacturing capacity is scarce, every GB of GDDR7 made for an AI inference card is competing with GDDR6 that could have gone into an AMD Radeon gaming GPU. TrendForce’s same December 2025 reporting noted that AMD is already feeling this squeeze in its Radeon lineup.

Standard DRAM: The Innocent Bystander

And then there’s good old DDR5 — the RAM in your laptop, your desktop, your server rack. It’s the most abundant, most affordable, least glamorous type of memory. It also shouldn’t be caught up in any of this.

But it is.

After a brutal market downturn in 2022–2023, the three major memory manufacturers — Samsung, SK Hynix, and Micron — cut production to stop prices from collapsing further. Then generative AI exploded. Demand for high-end memory went vertical almost overnight, and the manufacturers who had just tightened their belts suddenly couldn’t keep up. The ripple hit consumer memory hard. By November 2025, Tom’s Hardware was reporting — citing Japan’s Akiba PC Hotline — that retailers in Tokyo’s Akihabara electronics district had started limiting purchases: PC Shop ARK capped customers at 8 memory items per group; TSUKUMO cut it to 2 SO-DIMMs and 4 memory modules per buyer. By December 2025, TSUKUMO’s build-to-order brands, G-Gear and eX.computer, had stopped taking new orders altogether. Mouse Computers followed with its own suspension notice. This wasn’t a local Japanese quirk. It was a symptom of a global supply chain that had been quietly breaking for months.

The War Inside the War: Why HBM Is the Real Battlefield

So all three memory types are under pressure — but HBM is where the strategic fight is most intense. And it comes down to two problems that don’t have easy fixes.

The Number Micron Said Out Loud

The most important stat in this whole story isn’t a spec number. It’s a ratio — and it came straight from Micron’s own mouth.

In Micron’s Q2 and Q3 FY2024 earnings call prepared remarks, published on its investor relations page, the company stated plainly: “HBM3E consumes approximately three times the wafer supply as DDR5 to produce a given number of bits in the same technology node.” They added that the trade ratio for HBM4 is expected to be even higher. This isn’t analyst speculation. It’s a manufacturer telling investors — under disclosure obligations — exactly how their production allocation works.

What that means in practice: every time a fab shifts capacity to make HBM for an AI accelerator, it loses the ability to make roughly three times as much standard DRAM. It’s not a zero-sum trade in the sense that the tooling is perfectly interchangeable — HBM requires different process steps, and you can’t just flip a production line overnight. But the directional pressure is real and significant. The same wafer investment that serves AI’s premium needs is unavailable for the memory sitting in your next laptop.

The Bottleneck Nobody Talks About: Packaging

Here’s where it gets even more interesting — and more constrained.

Even if you have all the DRAM wafers in the world ready to become HBM, you still can’t ship them until they’re assembled. HBM stacks have to be bonded to processors using an advanced technique called CoWoS — Chip-on-Wafer-on-Substrate — a 2.5D packaging process that only a handful of facilities worldwide can do at scale. TSMC in Taiwan is the dominant player.

According to Next Platform’s detailed November 2024 reporting, TSMC’s CoWoS capacity was around 15,000 wafer packages per month in 2023. They scrambled to expand toward 45,000–50,000 by late 2024 — tripling capacity in roughly a year, which is no small feat. TrendForce projections in December 2025 pointed to 120,000 per month by the end of 2026. And yet demand is still outrunning supply.

This is the shift that industry professionals keep pointing to: packaging used to be an afterthought. It was the boring last step before shipping. Now it’s a strategic chokepoint — a narrow gate that the entire AI hardware industry has to pass through, regardless of how many wafers anyone can produce.

Three Companies, One Chokepoint

The HBM market is controlled by exactly three companies: SK Hynix, Samsung Electronics, and Micron Technology. SK Hynix has been the volume leader, supplying HBM3 and HBM3E to Nvidia for the H100, H200, and Blackwell GPU families. Micron’s HBM3E — running at a slightly faster 9.2 Gb/s signaling rate — was selected for specific H200 configurations, per Next Platform’s November 2024 reporting.

There’s a geopolitical dimension worth noting, too. Micron’s own CEO described the company as “the only US-based manufacturer of memory” in a formal earnings context in September 2025, per Blocks and Files reporting. CNBC had corroborated this structure as far back as 2023. The nuance: Micron’s leading-edge chips are still primarily made in Japan and Taiwan. Their first U.S.-soil leading-edge DRAM fab — a $15 billion facility in Boise, Idaho — is targeted for first wafer output in mid-2027. Until then, essentially all of the world’s advanced memory is made in East Asia by three companies, assembled through a single dominant packaging process. That concentration isn’t a conspiracy. It’s how decades of investment and specialization shake out. But it’s worth understanding when you’re thinking about supply chain risk.

The Ripple Effects You’re Already Living

This might feel abstract — wafer ratios, packaging bottlenecks, oligopolies. So let’s get concrete about what it means for actual people.

If you’ve tried to buy DDR5 recently and winced at the price, this is part of why. The 3:1 displacement ratio means that for every gigabyte of HBM going into an AI server, three gigabytes of consumer DRAM didn’t get made. Add the manufacturer capacity cuts of 2022–2023, which were supposed to stabilize the market, and you’ve got a perfect storm for buyers.

If you game on AMD — GDDR6 for Radeon cards is competing with GDDR7 for AI inference accelerators for the same fab time. When the allocation decision is made, the higher-margin AI product wins. TrendForce flagged this explicitly in their December 2025 reporting.

If you’re trying to run AI models yourself, the irony is that even the researchers and developers who need these accelerators can’t always get them. At Micron’s December 2025 earnings call, the company disclosed it was meeting only about 55–60% of core customer demand. One of the world’s three largest memory manufacturers, running flat out, can’t fill half its orders.

The Paradox That Won’t Go Away

Here’s something that keeps coming up in technical analysis of HBM — including SemiAnalysis’s detailed August 2025 report on the topic — and it’s a little mind-bending once you see it.

Every new generation of HBM delivers more capacity and more bandwidth. The H100 had 80 GB. The H200 jumped to 141 GB. The B200 was pushed to 192 GB. Each time, that feels like breathing room. Each time, AI developers immediately build models large enough to fill it — longer context windows, more parameters, and more aggressive use of memory that was previously too expensive. The techniques that squeeze models into tight memory budgets are relaxed the moment new headroom appears, only to be tightened again when the wall reappears.

Memory is a bottleneck that chases technology. It doesn’t get solved. It just moves.

What’s Coming Next: HBM4 and the Arms Race Continues

JEDEC — the standards body for memory — officially released the HBM4 specification in April 2025. The headline numbers: up to 2 TB/s of bandwidth per stack, a 2048-bit interface (double the previous generation), and stack heights supporting up to 64 GB per stack. Per Micron’s Q1 FY2026 earnings transcript, HBM4 is on track to ramp with high yields in Q2 2026, with pin speeds above 11 Gb/s.

AMD has already broken cover on what an HBM4 product looks like in practice. Their officially announced Instinct MI430X — part of the MI400 series, confirmed via AMD’s own November 2025 blog post — carries 432 GB of HBM4 at 19.6 TB/s of memory bandwidth, built on their new CDNA 5 architecture. It’s already slated for deployment in the Discovery supercomputer at Oak Ridge National Laboratory and Europe’s Alice Recoque exascale system. This isn’t a roadmap slide. It’s a real product with real deployments.

NVIDIA’s “Rubin” architecture is expected to follow with HBM4 as well, per Next Platform’s November 2024 reporting — though with GPU roadmaps, “expected” and “scheduled” are always doing a lot of heavy lifting.

The Bigger Picture: This Isn’t a Shortage. It’s a Realignment.

It’s tempting to read all of this as a temporary supply crunch — the kind that happens after every major demand shock, corrects itself in 18 months, and gets forgotten. But the structure here is different.

TrendForce data presented at the August 2025 Future Memory and Storage Summit shows HBM’s share of global DRAM wafer capacity rising from roughly 19% at the end of 2025 to roughly 23% at the end of 2026 — for HBM production alone. Layer GDDR7’s AI inference demand on top using the same equivalent-wafer methodology, and TrendForce’s December 2025 reporting projects total AI-attributable consumption approaching 20% of effective global DRAM manufacturing resources by 2026. To be precise: this is a manufacturing capacity metric expressed in equivalent wafer usage, not a simple share of bits shipped. The direction is not in question; the exact percentage is a well-grounded projection, not a certified measurement.

What is not a projection: in October 2025, OpenAI signed letters of intent with Samsung and SK Hynix — confirmed by OpenAI’s own announcement page, TechCrunch, and Bloomberg — for memory chip supply for the Stargate AI infrastructure project. Samsung and SK Hynix stated that OpenAI’s projected demand could reach 900,000 DRAM wafer starts per month as Stargate scales. No firm timeline was given for when that volume would actually be met. But the ambition of that number — relative to the roughly 160,000 HBM wafers per month SK Hynix was producing at the time — tells you something about the scale of what’s being built.

Major cloud providers, including Google, Amazon, Microsoft, and Meta, have similarly placed open-ended purchase commitments with memory suppliers — taking as much supply as is available, regardless of cost. When your customers are telling you, “we’ll take everything you can make,” you’re not in a normal market anymore.

Putting It Together

If you’ve made it this far, here’s the simple version:

HBM is the premium memory that powers AI accelerators — scarce, expensive, and bottlenecked not just by who can grow the crystals but by who can stack and bond the chips. That packaging process, CoWoS, is the narrow gate through which everything has to pass.

GDDR powers gaming GPUs and AI inference cards alike, and is quietly being squeezed as manufacturers chase higher-margin AI products over consumer graphics.

Standard DRAM powers the rest of computing and is being indirectly displaced by HBM’s 3:1 wafer trade ratio — a figure directly from Micron’s earnings transcript, not an analyst’s guess.

And the memory wall — the gap between how fast processors can compute and how fast memory can feed them — isn’t being solved. It’s being managed, one generation at a time, while developers build models large enough to exhaust whatever headroom each generation creates immediately.

The AI memory war isn’t a future scenario. It’s showing up in your RAM price today, in a Tokyo retailer’s purchase limits, and in an earnings call where one of the world’s three largest memory manufacturers admits it can’t come close to meeting demand.

There’s something almost poetic about the fact that the intelligence revolution is being bottlenecked not by ideas, not by algorithms, not even by the chips doing the math — but by the humble task of moving data from one place to another fast enough. Memory. The part of the computer most people never think about.

The next time you use an AI tool that feels slower than you expected, or you see a RAM price that makes you wince, or you read a headline about a chip company you’ve never heard of announcing a factory that won’t open until 2028 — you’ll know what’s actually happening underneath. A handful of companies in South Korea, the United States, and Taiwan are racing to build the plumbing for the most capital-intensive infrastructure project in human history. And right now, the pipes aren’t big enough.

If this kind of deep-dive is useful to you, I write about the infrastructure layer of AI — the hardware, supply chains, and economic forces most tech coverage skips over. Follow along here on [https://medium.com/@ainewblog007] so you don’t miss the next one.

And if you work in semiconductors, memory, or AI infrastructure, and I got something wrong — I genuinely want to know. The fact-check is always open.

This article originally appeared in my Substack newsletter, [https://substack.com/@imranvaliani]. Subscribe for new posts in this series.