Speculative Decoding: How to Get Free Tokens

LLM decoding is slow. Like embarrassingly slow. And if you’ve read my blog before this you know that the reason is memory. Generating one…