The second scaling law: the frontier moved to inference

From roughly 2018 to 2024, progress in AI ran on one reliable lever: make the pretraining run bigger. More parameters, more data, more…