DPO
Software / App
Direct Preference Optimization, a simpler RLHF algorithm that eliminates the need for a separate reward model and on-policy sampling. It works by taking gradient steps towards preferred responses and negative steps away from dispreferred ones.
Mentioned in 2 videos
Save the 2 videos on DPO to your own pod.
Sign up free to keep building your knowledge base on DPO as more episodes are added.
Videos Mentioning DPO
Stanford CS336 Language Modeling from Scratch | Spring 2026 | Lecture 15: Mid/Post-Training
Stanford Online
Direct Preference Optimization, a simpler RLHF algorithm that eliminates the need for a separate reward model and on-policy sampling. It works by taking gradient steps towards preferred responses and negative steps away from dispreferred ones.
Stanford CS329A Self-Improving AI Agents | Part 6 | Train Time Scaling/Scaling RL
Stanford Online
A typical reinforcement learning algorithm used in RLHF that involves keeping multiple policy models, critic, and reward models.