NCCL AllReduce Profiling on AWS g5 (4×A10G) — Nsight Systems Deep Dive

I recently profiled NCCL AllReduce performance on a 4× NVIDIA A10G (AWS g5.12xlarge) instance using NVIDIA Nsight Systems to understand the…