I Profiled LLM Inference From First Principles — Here’s What I Found

“Why is this so slow?” My colleague was frustrated. We work at the same company, and he was building a RAG system with a local LLM running…