Prefill/Decode Disaggregation: Why Your GPU Can’t Do Two Things at Once

I’ve been digging into the LLM inference space since I took my ML systems course at Carnegie Mellon University. Modern LLM inference has a…