14 years of AI: From Alexnet to Fable

Three of 2026’s biggest AI model launches — OpenAI’s GPT-5.6, Google’s Gemini 3.5, and DeepSeek’s V4 — are converging on the same two weeks in mid-July. That’s not a coincidence; it’s the endpoint of a competitive sprint that’s been compressing for years. To understand why the pace has gotten this frantic, and why nearly every frontier model today looks architecturally similar under the hood, it helps to walk back to where this all started: a single computer vision model in 2012 that could tell a cat from a dog.

Fourteen years later, models plan, code, and act across tools on their own. Here’s how we got from one to the other.

2012–2016: Before Transformers, CNNs Prove Deep Learning Works

Long before language models dominated the conversation, the breakthrough that made all of this possible happened in computer vision. In 2012, Alex Krizhevsky, Ilya Sutskever, and Geoffrey Hinton entered an 8-layer convolutional neural network called AlexNet into the ImageNet competition — and won by a landslide margin that stunned the field. That result is widely credited as the spark that kicked off the modern deep learning boom; it proved that with enough data and compute, deep networks could dramatically outperform the hand-engineered approaches that had dominated computer vision until then.

The years that followed were about making these networks deeper and more capable. In 2015, Microsoft introduced ResNet, which solved a critical problem: very deep networks tended to get worse, not better, as more layers were added. ResNet’s solution — skip connections that let information bypass layers — made it possible to train networks over 100 layers deep without that degradation, and the architecture became a foundation block used across the field for years afterward.

Meanwhile, a separate line of research was tackling sequences — translation, text generation, anything where order matters. Between 2014 and 2016, Google’s sequence-to-sequence (seq2seq) models, built on LSTMs (Long Short-Term Memory networks), produced the first genuinely useful neural machine translation systems. But these models had no attention mechanism yet. They processed text step by step, in order, which made them slow to train and prone to “forgetting” earlier parts of long sequences.

By 2016, the field had two separate toolkits: convolutional networks that excelled at vision, and recurrent networks that handled sequences reasonably well but couldn’t parallelize training or capture long-range context effectively. Both were about to be upended by a single idea.

2017–2018: One Paper Changes Everything

In 2017, a team at Google published a paper with an unassuming title — “Attention Is All You Need” — that introduced the Transformer architecture. Its core innovation was self-attention: instead of processing a sequence step by step like an LSTM, a Transformer could weigh every token in a sequence against every other token simultaneously. This was a fundamentally different way of handling sequences — parallelizable, scalable, and able to capture long-range relationships that recurrent networks struggled with. It’s genuinely difficult to overstate this paper’s importance: every major model discussed for the rest of this article, without exception, is a descendant of this architecture.

The Transformer immediately forked into two lineages. In 2018, Google released BERT, an encoder-only Transformer trained to understand bidirectional context — meaning it could look at words both before and after a given token to understand meaning. BERT dominated NLP benchmarks for years and became the standard architecture for tasks like search relevance and text classification.

The same year, OpenAI released GPT-1, a decoder-only Transformer built around a different idea: pretrain on a large amount of text, then fine-tune for specific tasks. This decoder-only, autoregressive approach — predict the next token, one at a time — is the direct ancestor of every GPT, Claude, Gemini, and Llama model that followed.

The architectural split matters: encoder-style models like BERT became specialists at understanding text, while decoder-style models like GPT became specialists at generating it. The generative lineage is the one that would go on to define the next decade of AI.

2019–2020: The Scaling Laws Era

With the architecture settled, the next open question was simple: what happens if you just make these models bigger? OpenAI’s GPT-2, released in 2019, had 1.5 billion parameters — large enough at the time that OpenAI initially withheld the full model, citing concerns it was “too dangerous” to release openly. By today’s standards, GPT-2 is a small, easily surpassed model, but the caution it prompted says something about how quickly the field’s sense of scale would shift.

That shift arrived in 2020 with GPT-3, a 175-billion-parameter model — more than 100 times larger than GPT-2. GPT-3’s headline capability was few-shot learning: it could perform new tasks from just a handful of examples in its prompt, without any task-specific fine-tuning. This was a genuine surprise to much of the field, and it validated something researchers had been theorizing: scale alone, without new architectural tricks, could unlock qualitatively new capabilities.

That intuition was formalized the same year by Jared Kaplan and colleagues at OpenAI in a paper on “scaling laws,” which showed that model performance improved in a smooth, predictable way as you increased parameters, training data, and compute together. For the next several years, this became the industry’s operating philosophy: if you want a better model, the most reliable lever is simply building a bigger one and training it on more data.

2022: The ChatGPT Moment

Scale alone wasn’t the whole story, though. In January 2022, OpenAI released InstructGPT, which introduced a training technique called Reinforcement Learning from Human Feedback (RLHF). Instead of just predicting the next word in a document, RLHF trained models to follow instructions and produce outputs that human raters actually preferred. This was a smaller architectural change than it might sound — the underlying Transformer was largely the same — but it changed how these models behaved in a way that mattered enormously for usability.

That behavioral shift became visible to the entire world in November 2022, when OpenAI released ChatGPT, built on a variant of GPT-3.5. It reached 100 million users within two months — the fastest consumer product adoption curve in history at that point, and the moment large language models stopped being a research curiosity and became a mainstream product category overnight.

The lesson from this era is worth sitting with: the biggest leap in perceived AI capability during 2022 didn’t come from a new architecture or a much bigger model. It came from an alignment and training technique. How you train a model, it turned out, mattered just as much as how big you built it.

2023: The Open-Source Floodgates Open

If 2022 was about one company’s product moment, 2023 was about the entire field catching up — and diverging into open and closed camps. Meta released LLaMA 1 in February, and although it was intended for restricted research access, the model weights leaked within days. That leak effectively created the open-weight LLM ecosystem as we know it today; within weeks, an entire community of fine-tunes, quantized versions, and derivative research sprang up around it.

OpenAI answered in March with GPT-4, a multimodal model and a substantial reasoning jump over GPT-3.5 — but kept it fully closed-weight. Anthropic, founded by former OpenAI researchers, released Claude 1 and Claude 2 between March and July, built around a distinct alignment philosophy called Constitutional AI, which trains models against an explicit set of written principles rather than relying purely on human raters’ preferences.

Meta followed its leaked LLaMA 1 with an official, commercially-licensed Llama 2 in July, giving businesses a legally clear path to build on open weights. In September, French startup Mistral AI released Mistral 7B, a small dense model that punched well above its parameter count and made the case that efficient training mattered as much as raw size. And in December, Google DeepMind shipped Gemini 1, built natively multimodal from the ground up rather than having vision bolted on afterward.

By the end of 2023, the AI landscape had split cleanly into two tracks — well-funded closed labs racing on capability, and an increasingly capable open-weight ecosystem racing on accessibility and cost — a split that persists today.

2024: Multimodal and Mixture-of-Experts Arrive

2024 was the year multimodality went from a differentiator to table stakes, and the year a new architectural idea — Mixture-of-Experts (MoE) — started reshaping how frontier models were built.

Meta’s Llama 3 and Llama 3.1, released between April and July, pushed the open-weight dense model ceiling to 405 billion parameters, becoming the largest openly available dense model at the time. OpenAI’s GPT-4o, released in May, moved in a different direction: native voice and vision handled by a single model in real time, rather than routing between separate specialized systems. Anthropic organized its Claude 3 family — Opus, Sonnet, and Haiku — around a tiering strategy, offering the same underlying capability set at different points on the cost-and-speed spectrum. And Google’s Gemini 1.5 Pro pushed context windows to 1 million tokens, a scale that was a genuine first for a production model.

Underneath these product-facing changes, Mixture-of-Experts architecture started going mainstream. The idea: instead of activating every parameter in the model for every single token, a router network selects a subset of specialized “expert” sub-networks to handle each token. The practical effect is that a model can have enormous total parameter counts — and therefore enormous total capacity — while keeping the actual computational cost per token much lower than a dense model of equivalent size. This tradeoff would become central to nearly every frontier release from this point forward.

January 2025: The DeepSeek Shock

If there’s a single inflection point on this entire timeline, it’s this one. In December 2024 and January 2025, Chinese AI lab DeepSeek released DeepSeek V3 and R1 — MoE-architecture models that were fully open-weighted and, according to DeepSeek’s own disclosures, trained at a fraction of the compute budget that Western frontier labs were reportedly spending on comparable models.

The market reaction was immediate and severe: DeepSeek’s release triggered roughly $1 trillion in tech stock losses on January 27, 2025, including approximately $600 billion wiped off NVIDIA’s market cap alone in a single day. Investors were re-pricing a core assumption — that frontier AI capability required frontier-scale compute spending — and DeepSeek had just demonstrated that assumption might not hold.

Architecturally, R1’s contribution was arguably even more significant than its efficiency story. It popularized large-scale reasoning-via-reinforcement-learning layered on top of an MoE base — training the model to “think” through a problem via extended chains of reasoning before producing a final answer, rather than relying on prompting tricks to elicit similar behavior. Within months, essentially every major lab — OpenAI, Google, Anthropic, Meta, xAI, Alibaba — had visibly accelerated its own reasoning-model and MoE roadmap. If you’re looking for the clearest “who influenced who” moment in this entire history, this is it.

2025: The Agentic Pivot

With reasoning and MoE now standard ingredients, 2025’s story became about models doing things, not just answering questions. Meta’s Llama 4 — released as Scout and Maverick in April — was the first Llama generation built on MoE, activating just 17 billion parameters per token despite having up to 400 billion total parameters. Alibaba’s Qwen3, also released in April, introduced hybrid reasoning: a single model that could toggle between fast “instant” responses and slower “thinking” mode depending on task difficulty, without needing separate model variants.

Anthropic’s Claude Sonnet 4 and 4.5 extended agentic coding capability and long-horizon tool use — the ability for a model to plan and execute multi-step tasks across many tool calls without losing coherence. xAI pushed pretraining scale aggressively with Grok 3 and Grok 4, leveraging its Colossus GPU cluster, with Grok 4 Heavy becoming the first model to score 50% on Humanity’s Last Exam, a benchmark specifically designed to be extremely difficult to saturate. And OpenAI’s GPT-5 unified what had previously been separate reasoning and non-reasoning model lines into a single model with internal routing.

By the end of 2025, “agentic” had gone from a buzzword to a baseline expectation: users increasingly expected models to use tools, browse the web, write and execute code, and carry out multi-step tasks with minimal supervision.

H1 2026: The Frontier Sprints

The first half of 2026 saw release cadence compress dramatically — models that once shipped a year apart now arrived weeks apart. Anthropic’s Claude Opus 4.7 and 4.8 pushed frontier reasoning and coding performance further. Google rebuilt its flagship from a new base with Gemini 3 Pro in November 2025, followed by Gemini 3.1 Pro in February 2026, extending context length and agentic coding capability. OpenAI’s GPT-5.5, released April 23 under the internal codename “Spud,” offered a 1-million-token context window and notable gains in agentic coding and scientific reasoning tasks.

The same week, on April 24, DeepSeek released a preview of V4-Pro — a 1.6-trillion-total-parameter MoE model activating just 49 billion parameters per token — timed, seemingly deliberately, to land the day after GPT-5.5. And xAI’s Grok 4.3, released April 30, added a 1-million-token context window, native long-video input, and always-on reasoning as a default behavior rather than an optional mode.

By this point, the architectural convergence across labs was nearly total: MoE plus tunable reasoning depth had become the default recipe. Dense, single-mode models — the norm just two years earlier — were now the exception rather than the rule.

July 2026: Where We Are Today

As of this writing, three of the year’s most significant releases are landing within days of each other. OpenAI’s GPT-5.6 previewed on June 26 as three durable tiers — Sol, Terra, and Luna — rather than a single flagship model, and reached general availability on July 9. This tiered-family approach marks a departure from OpenAI’s earlier one-flagship-per-generation pattern.

xAI, now operating as SpaceXAI following its merger with SpaceX, released Grok 4.5 on July 8 — its first model co-trained with coding startup Cursor, and pitched by Elon Musk as “Opus-class” performance at a lower price point. Anthropic’s Claude Fable 5 and Sonnet 5 both shipped in June, though Fable 5 was briefly taken offline in mid-June to comply with U.S. export control requirements before access was restored on July 1.

Meanwhile, Google’s Gemini 3.5 Pro and DeepSeek’s V4 stable release are both targeting July 17 — Gemini 3.5 following a deliberate delay so Google could retrain from scratch rather than fine-tune its prior generation, and DeepSeek V4 graduating from preview status to full general availability the same week. If both land on schedule, it will be one of the most concentrated weeks of frontier model competition the industry has seen.

What Actually Changed in 14 Years

Stepping back, three trendlines define this entire history.

Parameter scale moved from AlexNet’s roughly 65 million parameters to GPT-3’s 175 billion, to Llama 3.1’s 405 billion densely-activated parameters, to today’s MoE giants with total parameter counts in the 1–2 trillion range. But total parameters increasingly overstate what’s actually being computed per token — thanks to MoE routing, the active parameter count on any given forward pass has stayed far smaller than the headline totals suggest. Scale didn’t just get bigger; it got smarter about how it’s spent.

The lineage of influence is a clean, traceable chain: the Transformer forked into BERT-style encoders and GPT-style decoders. GPT-3 proved that scale alone unlocks new capability. InstructGPT and ChatGPT proved that alignment technique matters as much as scale. LLaMA’s leak opened the entire ecosystem to open-weight competition. And DeepSeek’s R1 forced every remaining lab to adopt reasoning-via-reinforcement-learning on top of MoE, nearly simultaneously. Each step didn’t just improve on the last — it changed what the next competitor had to build to stay relevant.

The frameworks that carried all of this are easy to overlook but essential to the story: PyTorch, Hugging Face Transformers, vLLM, llama.cpp, TensorRT-LLM, SGLang, LangChain, and the Model Context Protocol (MCP) — the tooling layer that turned research breakthroughs into deployable, servable, actually-usable systems.

In 2012, the state of the art was a model that could tell a cat from a dog. In 2026, models plan, code, and act across tools on their own, with new frontier releases now arriving in the same week rather than the same year. The architecture underneath has converged; the pace hasn’t slowed down at all.