Prefill, Decode, and TTFT: The Three Numbers That Should Drive Your Inference Engine Choice

When teams pick an LLM inference engine, the conversation usually starts and ends with one word: throughput. How many tokens per second can…