The Practitioner's LLM Curriculum ← Week 9 · Speculative Decoding
Interactive · Week 9 · Section 5

How to generate K tokens per forward pass.

Speculative decoding uses a small "draft" model to propose K candidate tokens, which the large "target" model verifies in parallel. Accepted prefix tokens become output; the first rejection gets replaced by the target's correct choice. Watch tokens stream in greedy mode (one per pass) and speculative mode (multiple per pass), and see how the speedup depends entirely on the draft-target acceptance rate. The catch: this is "speed without quality loss" — the output distribution is mathematically identical to greedy decoding from the target alone.

Mode
Acceptance rate (α) 0.70
Draft length (K) 4
Playback
Tokens generated
0
/ 60 target
Forward passes
0
target-model evaluations
Tokens / pass
average so far
Speedup vs greedy
tokens / pass

Generated output

Current step · what just happened

Press Play to begin generation.

What's happening here