Why Batching Hurts Reasoning (and Why Systems Still Do It)

Batching is one of the most effective optimizations in deep learning.
It increases arithmetic intensity, hides latency, and maximizes GPU…