New Week, New Molecular AI Models: BioEmu-1 and Evo 2

This past week, two new molecular AI models were released: BioEmu-1 and Evo 2. Since it seems like there are new models popping up every week, it can get a bit overwhelming. However, given that these two models are from big players, I thought I would take them for a spin and let you know my thoughts.

BioEmu-1

BioEmu-1 is a unique model from Microsoft Research in that it can predict molecular dynamics (MD) trajectories of a given input sequence. So, where AlphaFold and other folding models take in a string of amino acids and output 1 structure, BioEmu can produce multiple output conformations of your folded protein.

Note: BioEmu’s folded predictions are based on ColabFold using AlphaFold2-multimer’s weights, which I thought was quite interesting.

In the Microsoft blog post announcement, they show that the model can accurately emulate the equilibrium distribution of MD simulations. They also show that the model can be used as a tool to evaluate protein stability.

Figure 2 from their post: BioEmu-1 reproduces the D. E. Shaw research (DESRES) simulation of Protein G accurately with a fraction of the computational cost. On the top, we compare the distributions of structures obtained by extensive MD simulation (left) and independent sampling from BioEmu-1 (right). Three representative sample structures are shown at the bottom.

The repo includes a chignolin test peptide sequence, which outputs a .PDB file of the chain. Note that placement of the side chains is incorrect as the structure needs to be relaxed.

A sample output of the chignolin test sequence: GYDPETGTWG

This also outputs a .xtc trajectory file of conformations of this folded peptide equal to the number of samples requested. This allows you to view to the sampled conformations of the predicted structure in visualization tools like PyMOL.

10 samples of the chignolin peptide prediction

Links:

• BioEmu GitHub: https://github.com/microsoft/bioemu
• Preprint: https://www.biorxiv.org/content/10.1101/2024.12.05.626885

My Thoughts

Ease to run: ⭐⭐⭐

• Easy-to-use setup.sh script with a fairly small set of dependencies.
• The model’s checkpoint is small (~125MB) and is hosted on Hugging Face, making it easy to download.
• The command to run the prediction is very simple:

python -m bioemu.sample --sequence GYDPETGTWG --num_samples 10 --output_dir ~/test-chignolin

Note: I made a Docker image to make this part easier to run: Code / DockerHub

• After the trajectory sampling step above, they recommend running a sidechain reconstruction/relaxation step…which I found hard to run. They include code for the recommended *HPacker* tool, which uses a separate conda environment of far too many (conflicting) dependencies.

Quality of output: ⭐⭐⭐⭐

• Based on your --num_samples parameter, the output PDB file will have this number of predicted conformations. However, the conformations aren’t “temporal” in that each sample is at a different random place in time/space. (Compared to real MD simulations that output each frame as a subsequent conformation to the previous frame.)
• The output PDB is in a “backbone frame representation”, which means the sidechains are smushed onto the backbone and thus not in the right position. You need to run a subsequent tool to relax/pack them into place.

Evo 2

Evo 2 is a DNA language model, which is poised for long context modeling and design with a context length of up to 1 million base pairs. It was developed through a collaboration between the Arc Institute, universities (Stanford, Columbia, UC Berkeley), NVIDIA, and others.

This model is unique in that it can design DNA sequences with a lot of background information about exon-intron boundaries, transcription capabilities, binding sites, and epigenomic structure.

In their example notebook about BRCA1 variant effect prediction, they show how to load in a VCF-like table of SNVs:

…which are re-assembled into the mutated sequences:

chrom          17
pos      41276135
ref             T
alt             G
score   -0.372611
class    FUNC/INT
Name: 0, dtype: object
--
Reference, SNV 0: ...TGTTCCAATGAACTTTAACACATTAGAAAA...
Variant, SNV 0:   ...TGTTCCAATGAACTGTAACACATTAGAAAA...

Then, the model can predict the likelihood score for the nucleotide at each position and calculate the delta between the reference and variant. They then proport that this evo2_delta_score can be used to predict the variant’s effect.

Links:

• Evo 2 GitHub: https://github.com/ArcInstitute/evo2
• Preprint: https://arcinstitute.org/manuscripts/Evo2

My Thoughts

Ease to run: ⭐⭐⭐⭐

• There are multiple model sizes available (1B to 40B parameters) with either 8,192 or 1M nucleotide context lengths. The larger ones will obviously require multiple/enterprise GPUs (they specifically mention needing H100s), limiting the accessibility.
• The library is easy to install from their Git repo.
• The models are inferenced through familiar PyTorch/LLM commands.

import torch
from evo2 import Evo2

evo2_model = Evo2('evo2_7b')

sequence = 'ACGT'
input_ids = torch.tensor(
    evo2_model.tokenizer.tokenize(sequence),
    dtype=torch.int,
).unsqueeze(0).to('cuda:0')

layer_name = 'blocks.28.mlp.l3'

outputs, embeddings = evo2_model(input_ids, return_embeddings=True, layer_names=[layer_name])

print('Embeddings shape: ', embeddings[layer_name].shape)

Quality of output: ⭐⭐⭐

• The sequence scores are based on likelihoods, which I think are useful. However, I’m not sure likelihoods directly translate to variant effect severity (rather, just disruption capability or how uncommon a nucleotide is in a given location).

They report an AUROC of 0.73 for the zero-shot variant effect predictions of BRCA1

• It appears that these models were trained on “8.8 trillion tokens from all domains of life”, though it would be useful to tune the models to a specific species of interest (e.g., humans).
• The model allows you to generate new nucleic acid sequences, though it's not clear how to conditionally generate sequences for a specific purpose or based on a reference set of genes, for example.

output = evo2_model.generate(prompt_seqs=["ACGT"], n_tokens=400, temperature=1.0, top_k=4)

Some Thoughts

Both of these new models, though different in purpose, are very well done. The quality of the code and instructions to get things running are quite good. As with any general model, I think we’ll see more utility in the future in fine-tuning for specific purposes (e.g., antibody MD predictions, oncology variant effect modeling). However, I think structural biologists will really like these models.

• For BioEmu, I’m excited to use this model to predict trajectories of some of my diffused antibody sequences, which may help understand molecular stability.
• For Evo 2, I could see myself using this tool to better understand mutations in infectious diseases (like in my current H5N1 avian influenza research).

If you or your organization would like help exploring these molecular AI models, please reach out: colby@tuple.xyz ✉

Stay curious…