Your Optimizer Spends Its Whole Life One Step From Exploding. On Purpose.

There is a precise speed limit above which gradient descent flies apart. You would expect a well-behaved optimizer to stay safely below it.