Initialization, BatchNorm, and LayerNorm: The Holy Trilogy

There are a ton of blog posts out there breaking down both initialization and normalization. However, I feel like most authors fail to…