LLM Quantization Under the Hood: Fitting LLaMA 3.1

A senior systems perspective on squeezing 16GB models into 5GB of VRAM without breaking accuracy.