⚡ Speculative Decoding • Draft Models • Medusa Heads • vLLM & TensorRT-LLM
Speculative Decoding Speedup Calculator
Model exact inference throughput acceleration when coupling a large target model (e.g. Llama-3.3-70B) with a lightweight speculative draft model (e.g. Llama-3.2-1B / 3B).
Model Pair Configuration
5 tokens
Number of speculative tokens proposed by the draft model per verification round.
72%
Probability that target model accepts draft token under exact distribution matching.
Net Speedup Ratio
2.41x
Throughput boost
Accepted Per Round
3.92 tok
Average yield / verification
Effective Latency
16.6 ms
Per token (was 40.0 ms)
Latency Timeline Comparison (100 Generated Tokens) 2.34s saved
Standard Autoregressive Target Model 4,000 ms
Speculative Decoding Pipeline 1,660 ms
📐 Theoretical Verification Formulation:
E[Tokens / Step] = (1 - α^(gamma+1)) / (1 - α) • Speedup S = E[N] / [ (gamma • t_draft + t_target) / t_target ]
Because speculative decoding preserves 100% mathematical token distribution parity through modified rejection sampling, target model quality remains completely lossless.
⚙️ Production Engine Configuration (vLLM / TensorRT-LLM)
Copy this parameter setup for high-throughput serving: