From AlphaFold to ESM3: The Era of Programmable Biology

The biological world is governed by the Central Dogma: a 1D amino acid sequence dictates a complex 3D structure, which ultimately defines a protein’s function. Deciphering this sequence-to-structure relationship has historically been one of the grandest challenges in computational biology.

The sheer scale of the problem is difficult to overstate. Because each amino acid in a chain can twist and bend at multiple angles, the number of potential 3D shapes a single protein can adopt is staggeringly large. To put this into perspective, a standard protein could theoretically fold into 10^300 different conformations, a number vastly exceeding the total number of atoms in the observable universe. If a protein had to randomly test every possible shape to find its correct native structure, it would take longer than the lifespan of the universe. Yet, in nature, proteins snap into their precise 3D shapes in mere milliseconds!

To understand why early physics-based computational models often failed, one must visualize protein folding not as a straightforward descent, but as navigating a remarkably rugged and treacherous energy funnel. As the integrated folding and aggregation landscape illustrates, a protein begins at the wide rim of the funnel as an unfolded ensemble, possessing high free energy and vast conformational freedom. Early computational tools attempted to fold proteins through pure global energy minimization, simulating atomic physics to pull the structure down toward the “Native State” at the absolute bottom (the global minimum). However, this mathematical terrain is infinitely complex, riddled with kinetic traps, shallow local minima, and competing deep valleys, such as the sometimes irreversible plunge into misfolded amyloid aggregates. Because early algorithms relied primarily on localized gradient descent without the broader macroscopic map provided by evolutionary co-mutation data, they essentially explored this dark, jagged canyon blindfolded; they frequently became stuck in local energetic traps or computationally overwhelmed by the sheer dimensionality of the landscape before ever reaching the true native structure.

Today, we are witnessing an evolutionary leap in how AI navigates this hyper-astronomical search space, transitioning from the groundbreaking AlphaFold models to Evolutionary Scale Modeling (ESM3). This progression is officially ushering in the era of “programmable biology.”

A Brief History of Protein Folding

The quest to solve the protein folding problem has evolved over several decades:

• 1972 (Anfinsen’s Dogma): It was established that a protein’s amino acid sequence natively determines its structure.
• 1994 (CASP Founded): The Critical Assessment of Protein Structure Prediction (CASP) was formed to rigorously benchmark algorithmic progress in the field.
• 2000s (Physics-Based Models): Tools like Rosetta relied heavily on energy minimization techniques to predict folding.
• 2016 (Deep Learning Integration): The modern era began with the first use of Convolutional Neural Networks (CNNs) on co-evolutionary maps.

The Deep Learning Era and AlphaFold

The pivot to deep learning was fueled by a massive data explosion. As the cost of DNA sequencing plummeted, databases like UniProt experienced exponential growth, providing the raw “fuel” needed to train massive AI models. Algorithms began leveraging Multiple Sequence Alignments (MSAs) to identify co-evolutionary signals. The logic was simple but powerful: if two amino acid residues mutate together over evolutionary time, they are highly likely to be touching in physical 3D space. This set the stage for the AlphaFold breakthroughs.

AlphaFold 1 (2018): Won the CASP13 competition by successfully combining MSAs with deep CNNs. Its core architecture relied on dual representations: an MSA representation to process evolutionary history and co-mutation, and a Pair representation (a 2D matrix) to capture spatial relationships and geometry between amino acids.

**AlphaFold 2 (2020): **While AlphaFold 1 was a breakthrough, it still relied on legacy deep learning architectures and indirect predictions. AlphaFold 2 completely rebuilt the system from the ground up, moving away from Convolutional Neural Networks (CNNs) to an Attention-based architecture, ultimately solving the folding problem for single chains with atomic-level accuracy. The massive leap in performance came down to three major architectural innovations:

• The Evoformer and “Crosstalk”: AlphaFold 1 processed evolutionary data (MSAs) and spatial data (pairwise amino acid interactions) somewhat independently using CNNs. CNNs are great for local patterns (like adjacent pixels in an image), but struggle with long-range dependencies, like two amino acids that are far apart in the sequence but fold to touch each other in 3D space. AlphaFold 2 introduced the Evoformer, a Transformer-based block that continuously exchanges information between the 1D MSA representation and the 2D Pair representation. As the model updates its understanding of evolutionary mutations, it instantly updates its understanding of spatial geometry, and vice versa. This dynamic “crosstalk” allows the model to deeply reason about spatial constraints.
• Bypassing Distance Maps for Direct 3D Coordinates: AlphaFold 1 did not actually output a 3D protein. It outputted a “distogram”, a 2D matrix predicting the probability of distances between amino acids. Researchers then had to use a separate, physics-based optimization program to fold a 3D structure that matched those distances. AlphaFold 2 became fully end-to-end. It introduced a specialized Structure Module equipped with Invariant Point Attention (IPA). Instead of just predicting distances, this module reasons natively in 3D Euclidean space, directly manipulating a “gas” of amino acids and rotating/translating them to output exact 3D Cartesian coordinates (X, Y, Z) for every atom.
• Iterative Recycling: AlphaFold 2 introduced a concept called “recycling.” Instead of doing a single pass, the model takes its generated 3D structure and feeds it back into the beginning of the Evoformer network. By looping the output back through the system multiple times (usually 3 or 4 passes), the model can iteratively refine its own predictions, fixing structural clashes and fine-tuning atomic positions much like a human double-checking and revising a complex math problem.

The impact was monumental. AlphaFold 2 provided highly accurate structural models for nearly all known proteins, accelerating biological research globally!

The Rise of Protein Language Models (pLMs)

Despite AlphaFold’s success, an “MSA bottleneck” remained. Generating MSAs is computationally expensive and frequently fails for “orphan” proteins that lack known evolutionary relatives. Enter Protein Language Models (pLMs)!

[embed]Science | AAAS © 2026 American Association for the Advancement of Science. All rights reserved. AAAS is a partner of HINARI, AGORA…www.science.org

To understand why pLMs are so effective, we must look at the “data as language” paradigm. By treating a protein sequence as a sentence and individual amino acids as “words” or “tokens,” researchers realized they could apply the exact same Transformer architectures that power massive language models like ChatGPT.

Transformers are fundamentally generalists; they do not inherently “know” English, code, or biology. Instead, they are master pattern-recognizers that learn the statistical relationships and contexts between tokens. If you feed a Transformer billions of protein sequences, it doesn’t just memorize them, it learns the underlying biological grammar of life. These models possess incredible zero-shot power, meaning they can predict both structure and function purely from a single sequence, offering massive scalability without needing an MSA.

Introducing ESM3: A Unified Multimodal AI for Biology

Developed by researchers at EvolutionaryScale, ESM3 represents the latest frontier. It is a massive multimodal generative language model that natively reasons across sequence (1D code), structure (3D atomic coordinates), and function (biological properties) simultaneously. Trained on 2.78 billion proteins and boasting 98 billion parameters, the sheer scale of ESM3 is staggering.

This brings us to the true power of the Transformer architecture: its ability to unify disparate types of data by treating everything as language. Just as Vision Transformers (ViTs) revolutionized computer vision by chopping images into “patches” and feeding them to the model as a sequence of tokens, ESM3 applies this same abstraction to the physical world of proteins.

To achieve this, ESM3 uniquely tokenizes biology:

• Discrete Vocabulary: ESM3 translates continuous biological data into a discrete vocabulary that a Large Language Model can read and write.
• The 3D Token: Using a trained discrete autoencoder, 3D local atomic structures are compressed into a finite vocabulary of structural tokens. Just like a Vision Transformer turns pixel patches into tokens, ESM3 turns physical shapes into tokens. This allows the model to “speak” in protein geometries just as easily as it speaks in amino acid letters.
• Track Fusion: At the input level, sequence, structure, and functional keywords are embedded and fused into a single unified latent space. Because everything has been reduced to a common “language” of tokens, the Transformer can process them together seamlessly using its universal attention mechanism. Translating a 1D sequence into a 3D structure becomes conceptually identical to translating English into French.

Additionally, ESM3 incorporates a Geometric Attention Layer. Inserted into the first Transformer block, this layer directly conditions attention weights on the actual 3D coordinates of the protein’s atomic backbone, deeply grounding the model in physical reality.

Programmable Biology and The GFP Challenge

ESM3 was trained via Generative Masked Modeling. By taking natural proteins and masking out varying combinations of sequence, structure, or function tokens, the model was forced to predict the missing pieces.

Because data is processed “all-to-all”, researchers can now use Iterative Unmasking. A scientist can prompt the model with a partial biological scaffold or functional keywords, and ESM3 will fill in the masked tokens step-by-step to design a completely novel protein around the prompt. It acts as a highly steerable biological co-pilot.

To test its generative limits, the team attempted The GFP Challenge. Green Fluorescent Protein (GFP) features a highly complex 11-stranded beta-barrel and a spontaneously forming light-emitting chromophore, a true “needle in a haystack” in the vastness of protein space.

The result was a resounding success. The model designed esmGFP, which was synthesized in the lab and worked perfectly on the first try. The AI accurately assembled the complex beta-barrel and self-forming chromophore exactly as prompted. Astonishingly, the resulting sequence shared only 58% sequence identity with natural proteins and contained 96 mutations, a generative leap mathematically equivalent to simulating 500 million years of evolution.

Implications for the Future

Bridging the gap between sequence, structure, and function by treating biology as a programmable language enables the generation of completely novel proteins. The future implications of this technology are vast, some of which are:

• Therapeutics: The on-demand design of incredibly precise, targeted antibodies.
• Climate: The engineering of novel enzymes explicitly designed for rapid plastic degradation and atmospheric carbon capture.
• Materials: The creation of new, sustainable biomaterials with exceptional strength and durability.

We have officially moved past simply predicting biology into an era where we can actively program it!

This blog is written based on a talk recently given by me at Indian Institute of Science, Bangalore.