KL Divergence
A metric used in knowledge distillation to measure the difference between two probability distributions. It's employed to train smaller models to approximate the output distribution of larger models.
Save the 4 videos on KL Divergence to your own pod.
Sign up free to keep building your knowledge base on KL Divergence as more episodes are added.
Videos Mentioning KL Divergence
![[Paper Club] Molmo + Pixmo + Whisper 3 Turbo - with Vibhu Sapra, Nathan Lambert, Amgadoz](https://i.ytimg.com/vi/8BN9CdIYaqc/maxresdefault.jpg)
[Paper Club] Molmo + Pixmo + Whisper 3 Turbo - with Vibhu Sapra, Nathan Lambert, Amgadoz
Latent Space
A metric used in knowledge distillation to measure the difference between two probability distributions. It's employed to train smaller models to approximate the output distribution of larger models.
The Origin and Future of RLHF: the secret ingredient for ChatGPT - with Nathan Lambert
Latent Space
A distributional distance used as a constraint in RLHF objectives, acting as a guardrail to prevent overfitting to small datasets and maintaining model stability.
Foundations of Unsupervised Deep Learning (Ruslan Salakhutdinov, CMU)
Lex Fridman
Used in variational learning to measure the difference between an approximating distribution (recognition model) and the true posterior.
Stanford CME296 Diffusion & Large Vision Models | Spring 2026 | Lecture 1 - Diffusion
Stanford Online
A measure of how one probability distribution differs from a second, reference probability distribution, used to quantify the difference between the modeled and target distributions.