From SGD to LAMB: A Deep Engineering Walkthrough of Modern Optimizers

Why we moved from “just gradient descent” to layer-wise trust ratios — and what each step actually fixes.