The MoE Latency Trap: Why Mixture-of-Experts Loses at Low Concurrency

Everyone benchmarks MoE at peak throughput. I benchmarked it at batch=1.