DISTRIBUTED TRAINING: WHAT I LEARNED BUILDING A PYTORCH TRAINING SYSTEM FROM SCRATCH

More often than not, I wake up with an undying thirst for knowledge. On a faithful morning weeks ago, I woke up wanting to understand what…