Distillation
The process of using larger AI models as 'teacher' models to train smaller, more efficient 'student' models.
Save the 4 videos on Distillation to your own pod.
Sign up free to keep building your knowledge base on Distillation as more episodes are added.
Videos Mentioning Distillation
The Unreasonable Effectiveness of Reasoning Distillation: using DeepSeek R1 to beat OpenAI o1
Latent Space
A training technique where a smaller model learns from the outputs (logits or generated data) of a larger, more capable 'teacher' model.
The 10 Trillion Parameter AI Model With 300 IQ
Y Combinator
The process of using larger AI models as 'teacher' models to train smaller, more efficient 'student' models.
Inference, Diffusion, World Models, and More | YC Paper Club
Y Combinator
A method to transfer knowledge from a larger model or ensemble to a smaller model, reducing inference compute.
Stanford CME296 Diffusion & Large Vision Models | Spring 2026 | Lecture 8 - Trending Topics
Stanford Online
A category of methods aimed at shortening the number of inference steps needed to generate samples from a model, thereby reducing generation time and computational cost.