Why Code, Why Now: An Information-Theoretic Perspective on the Limits of Machine Learning

TL;DR: Code generation has succeeded where reinforcement learning struggles to accumulate general competence: not mainly because models got…