Smaller, Slower, Wrong: What Aggressive Quantization Costs On-Device Inference

On my Pixel, the most compressed model I built ran slower than the bigger one, and it thought a dog was a shower curtain