Why SGD is a Secret Regulariser: An Empirical Deep Dive

The optimizer you’ve been using every day is doing far more than just minimizing your loss function.