A small draft model guesses a few tokens, a large target model checks them all in one forward pass, and the text comes out exactly as the large model alone would write it. The playground runs that loop on real models and lets you inspect every round.
Which tokens did the draft get right?
Run speculative decoding on your own prompt with real models, in sampling or greedy mode, and see the output colored by where each token came from.
What happened inside one round?
Every round broken into five steps, draft, verify, accept or reject, correction or bonus, and commit with cache rollback, with live numbers and the lines of code that ran highlighted.
Was it actually faster?
Where the time went in each run, checked against the target model decoding on its own with the same seed.
Why is it exact, and when does it pay off?
Drag two toy distributions to see why accepting with probability p/q reproduces the target exactly, then explore the speedup as a function of acceptance rate, cost ratio and γ.
Speculative decoding is easy to describe and easy to get subtly wrong. A bug in the acceptance test, the residual distribution or the cache rollback still produces fluent text; it just isn't the large model's text any more. Reading the papers rarely shows where those mistakes hide.
The playground runs a complete reference implementation of about a hundred lines, shows that code next to every step, and is tested to emit exactly the same tokens as the reference and, in greedy mode, exactly the target model's own output.
A Python server loads a draft and a target model that share a tokenizer, runs the decoding loop with read-only snapshots of every intermediate value, and sends the whole trace to a plain HTML and JavaScript page with no build step.
The live server runs SmolLM2-360M as the target and SmolLM2-135M as the draft. At this size every forward pass costs about the same fixed overhead, so the draft is not much cheaper than the target and there is little speedup; the page measures and explains this. The gain appears when the target is large and memory-bound, which the second lab models.
Speculative Decoding Playground stands on open-source work and published research. Thank you to everyone behind it.
A set of labs, walked in order, that run real TransformerLens models behind a web page. Type a prompt, pick a model, and see how it is tokenized, where every head attends, and when the model knows its answer.
A tiny real transformer (d = 8, 2 heads, an 8 → 32 → 8 MLP, 4 blocks) runs on the sentence you type. Every circle holds the number it actually computed, and hovering one shows the weighted lines that feed it.
One concrete configuration, followed from start to finish: a 1.27B-parameter model with Llama 2 7B's layer shape, trained on a single 8-GPU node split 2-way tensor × 2-way pipeline × 2-way data parallel, then post-trained and deployed.