Distillation
Concept
The process of using larger AI models as 'teacher' models to train smaller, more efficient 'student' models.
Mentioned in 2 videos
Videos Mentioning Distillation
The Unreasonable Effectiveness of Reasoning Distillation: using DeepSeek R1 to beat OpenAI o1
Latent Space
A training technique where a smaller model learns from the outputs (logits or generated data) of a larger, more capable 'teacher' model.
The 10 Trillion Parameter AI Model With 300 IQ
Y Combinator
The process of using larger AI models as 'teacher' models to train smaller, more efficient 'student' models.