Your GPUs are fast. Your paths between them might not be.

Identical GPUs, training loop, but different throughput — check the physical paths data takes between cards.