Float16 vs Float32: What GEMM and My Benchmarks Taught Me About GPU Performance

When I first started benchmarking matrix multiplication on my system, I thought I already understood what was going to happen. Float16 uses…