Progressive Distillation
Concept
A distillation technique where a student model is trained by progressively working on problems of approximate constant difficulty, halving the number of steps at each iteration to reach a single-step generation.
Mentioned in 1 video
