The Art of Guessing Fast: Speculative Decoding & Speculative Speculative Decoding

From tokens to Saguaro, a beginner-to-expert guide to the most clever inference trick in LLMs right now ; Based on: Leviathan et al. 2023 +…