What is Grokking?

Introduction

Grokking is a surprising phenomenon observed in neural network training where a model first memorizes data without truly understanding it, and then after long training suddenly begins to generalize correctly on new examples. It was first reported in research by Power et al. (2022) and has since become an important topic in modern AI interpretability.

What makes grokking interesting is that it reveals about how learning actually unfolds inside neural networks.

From Overfitting to Generalization

What researchers observe

When training certain models on structured tasks (like modular arithmetic or rule-based datasets), a consistent pattern is observed:

• First, the model learns to fit training data very well
• But it fails on unseen data (overfitting)
• Then, after continued training, performance on new data suddenly improves

This delayed improvement is what researchers call grokking.

What Is Happening Inside the Model

Two phases of learning

Researchers studying this phenomenon describe two distinct internal behaviors:

1. Memorization phase

In the early stage, the model behaves like a lookup system. It stores patterns tied closely to training examples without capturing the exact rule behind them.

2. Structure discovery phase

With continued training, the model gradually shifts toward simpler internal representations. Instead of memorizing examples, it begins to form more general rules that explain the data.

This phase is slow internally but appears sudden when measured by test performance.

The Role of Interpretability Research

How researchers study grokking

Scientists like Neel Nanda and others in mechanistic interpretability try to understand what changes inside the network during grokking. They use methods such as:

• circuit analysis
• activation patching
• tracking weight evolution

Source Image

Their findings suggest that grokking is the reorganization of how information is represented inside the model.

Grokking as a Change in Internal Representation

Varma et al.’s work on interpretability does not focus on grokking directly, but it provides the foundation for understanding it. His key idea is:

Neural networks can be understood as systems of internal circuits and features. This perspective allows researchers to treat grokking as a change in internal structure. It makes it possible to ask how the model represents information, not only whether it gets the answer right.

Scaling Perspective and Training Dynamics

Why grokking is not always expected

From a broader machine learning perspective (including ideas linked to Jared Kaplan’s work on scaling laws), model performance is often expected to improve smoothly with training.

Grokking challenges this intuition. Instead of smooth improvement, the model shows a long plateau followed by sudden generalization.

This means, optimization does not always find the best solution ralidly. Sometimes it explores inefficient representations first before settling into a more structured one.

Why It Happens

Grokking is influenced by:

• long training duration
• weight decay
• small or structured datasets
• optimization dynamics of gradient descent

Together, these factors allow the model to eventually “prefer” simpler internal rules over memorization.

Why Grokking Matters

Grokking shows that:

• good generalization may appear late, not early
• neural networks can reorganize internally
• training time can fundamentally change learned behavior

It provides a rare window into how internal representations evolve, helping researchers understand how complex behavior appears from simple optimization rules.

Conclusion

Grokking is not a jump in intelligence. It is a delayed transition in learning where a model moves from memorizing examples to discovering structure.

Seen through modern interpretability research, it reflects that neural networks do not just learn answers. But, they gradually reorganize how they think about a problem. And sometimes, that reorganization only becomes visible after a long silence in performance.