The ML Intuition

The Logistics of Large-Scale Training: DP, TP, PP, and the Role of Communication

When I first started working on distributed training, I didn’t realize how much was happening behind the scenes. To put it simply — I was…