[Paper Review] TRM

Less is More: Recursive Reasoning with Tiny Networks

The ultimate goal of artificial intelligence is to build an AGI. To build an AGI, reasoning including deduction is crucial. Researchers tried various ways to make models reason effectively and correctly. Currently, the hierarchical reasoning model (HRM) has made a novel approach using two small neural networks recursing at different frequencies. Inspired by this, the researcher made TRM, a Tiny Recursive Model, a much simpler recursive reasoning approach.

Introduction

LLMs, including ChatGPT, Gemini, and Claude, are strong, but they still struggle to solve hard question-answer problems. Most LLMs rely on Chain-of-Thoughts(CoT) using prompt engineering, mimicking humans’ thought process. Also, Test-Time Compute(TTC) is used to improve the reliability of the answers. However, these two approaches are brittle and lack long-term application.

An alternative direction named HRM was proposed, which improves hard tasks such as ARC-AGI, Sudoku solving, and Maze pathfinding. HRM has two novelties: recursive hierarchical reasoning and deep supervision. The researchers of HRM used two networks and made the model reason recursively. They provide biological arguments that the brain operates and processes hierarchically using two different frequencies.

Deep supervision consists of improving the answer through multiple supervision steps. In prior studies, deep supervision improved the accuracy twice a single step of supervision. However, recursion for the supervision step is not important.

Background

HRM has 4 trainable features: input embedding, low-level recursive network, high-level recursive network, and result head. Also, HRM has four main features. First is recursion at two different frequencies. The high-level frequencies and low-level frequencies are recursing to make a reasonable answer. Second is fixed-point recursion. Assuming that (zL, zH) reaches a fixed point (zL, zH) through recursing high-level frequencies and low-level frequencies, a 1-step gradient approximation approximates the gradient by back-propagating. Third is deep supervision. Deep supervision is used to increase the effectiveness in depth. The researchers reuse the previous latent features as initialization for the next forward pass. Finally, Adaptive Computational Time(ACT) is applied, which uses Q-learning to pass the zH through an additional head and run an additional forward pass.

Tiny Recursion Models

The author of “Less is More: Recursive Reasoning with Tiny Networks” improved the HRM approach while minimizing the computational usage. Unlike HRM, Tiny Recursion Models use a single tiny network and a single forward propagation for ACT.

The author reinterprets the two latent features, zH and zL, as the current solution and latent reasoning state rather than biologically inspired hierarchies, showing that having both is necessary for stable recursive improvement.

TRM doesn’t require mathematical(fixed-point theorem) or biological explanations. Also, it uses only a single network. The researcher empirically found that 2 layers are better than bigger layers because the bigger models are prone to overfitting. ACT is simplified by replacing Q learning with binary cross-entropy-based halting loss, reducing the forward propagation to once while preserving the performance.

Furthermore, TRM is attention-free. For small, limited context length, self-attention is inefficient. The researcher replaced self-attention with a multi-layer perceptron. In the case of Sudoku solving, MLP was efficient than self-attention, while with a larger context, self-attention is better.

Conclusion

Overall, TRM simplifies HRM by removing unnecessary complexity while achieving higher performance. It requires no fixed-point theorem, no multi-network setup, and achieves strong generalization with tiny architectures.