Speculative Sampling

Concept

A lossless method for speeding up inference by using a cheaper draft model to generate multiple tokens, which are then verified by a larger target model in parallel.

Mentioned in 1 video

Videos Mentioning Speculative Sampling

Stanford CS336 Language Modeling from Scratch | Spring 2026 | Lecture 10: Inference

Stanford CS336 Language Modeling from Scratch | Spring 2026 | Lecture 10: Inference

Stanford Online

A lossless method for speeding up inference by using a cheaper draft model to generate multiple tokens, which are then verified by a larger target model in parallel.