Nature Methods Review | Beginner’s Tutorial on Transformer Models: Spatial Transcriptomics/Single-Cell/Genomics

Preface:​ This paper reviews the applications of Transformer​ models in analyzing three common types of genomic data — genome sequences, single‑cell genomic data, and spatial transcriptomics data — and provides corresponding beginner tutorials.

Transformer‑based models are rapidly becoming foundational tools for analyzing and integrating multi‑scale biological data. The core idea of the Transformer is to take a sequence of symbols (such as words in a sentence or nucleotides in DNA) as input, and learn the patterns in which each symbol typically appears within a given context. The first step is tokenization, during which the input sequence is split into smaller units (e.g., words, subwords, or nucleotides) and mapped to unique tokens in a predefined vocabulary. The self‑attention mechanism​ enables the Transformer to identify which combinations of tokens are critical for a particular task. In this way, the Transformer captures the “grammar” needed to solve different tasks by determining the relative importance of different parts of the symbol sequence within their context.

On December 1, 2025, the team led by Jesper Tegner at King Abdullah University of Science and Technology (KAUST), Saudi Arabia, published a review article in Nature Methodstitled “Multimodal foundation transformer models for multiscale genomics.”​ The paper reviews recent advances in Transformer architectures, tracing their evolution from single‑modal and enhanced single‑modal models to large‑scale multimodal foundation models capable of operating on genome sequences, single‑cell transcriptomics, and spatial data. The authors classify these models into three hierarchical levels and evaluate their capabilities in structural learning, representation transfer, as well as tasks such as cell annotation, prediction, and imputation.

When discussing challenges such as tokenization, interpretability, and scalability, the paper highlights recent approaches leveraging masked modeling, contrastive learning, and large language models. To promote broader adoption, practical guidance is provided through code‑based beginner tutorials, using public datasets and open‑source implementations. Finally, the authors propose designing a modular “super Transformer” architecture via cross‑attention mechanisms to integrate heterogeneous modalities.

URL: https://www.nature.com/articles/s41592-025-02918-6