Scaling Deep Learning with Distributed Training — Part II(a): How GPUs talk ?

This post is a small novella in our Distributed Training series — think of it as the “Dunk & Egg” to our “House of the Dragon.”