Key Moments
Stanford CS229 Machine Learning | Spring 2026 | Lecture 11: Diffusion Models
Stanford OnlineWant to know something specific about what's covered?
We've already dissected every moment. Ask and we will deliver (with timestamps).
Key Moments
Diffusion models generate photorealistic images by gradually denoising random noise over many steps. While highly effective, the process is computationally intensive, requiring thousands of steps for high-quality generation.
Key Insights
Diffusion models are now the predominant approach for image generation, surpassing GANs and variational autoencoders in effectiveness.
The forward diffusion process involves gradually adding Gaussian noise to an image, with the noise variance controlled by a schedule of small values (beta_t) between 0 and 1.
The reverse diffusion process, which generates an image from noise, is parameterized as a Gaussian distribution whose mean is a neural network dependent on the noisy image and the current timestep (t).
The training objective is derived from the Evidence Lower Bound (ELBO) and decomposes into a sum of per-step KL divergences, effectively becoming a denoising score matching objective.
The key training loss simplifies to minimizing the difference between the predicted noise and the actual noise added at each step, scaled by the noise schedule.
A theoretical justification for using Gaussian distributions in the reverse process stems from continuous-time stochastic differential equations, where the reverse process is proven to be Gaussian in the limit.
The dominance of diffusion models in generative tasks
Diffusion models have emerged as the leading technique for generative tasks, particularly in image generation. They significantly outperform previous methods like Generative Adversarial Networks (GANs) and Variational Autoencoders (VAEs). This advancement has led to their adoption in generating not only images but also videos and actions for robotics. There's even research exploring their application in natural language processing to accelerate large language models, although the core focus remains on image synthesis.
The forward diffusion process: gradually adding noise
The core idea of diffusion models lies in a two-stage process: a fixed forward diffusion process and a learned reverse denoising process. The forward process systematically corrupts data by gradually adding Gaussian noise over a series of timesteps, say from T=0 to T=max. Starting with a clean image X0, at each step t, a small amount of Gaussian noise is added. This is mathematically represented as X_t = sqrt(1 - beta_t) * X_{t-1} + sqrt(beta_t) * epsilon_t, where epsilon_t is standard Gaussian noise and beta_t is a small variance schedule, typically between 0 and 1. Over many steps, X_t progressively becomes pure noise. A key property derived from this formulation is that X_t can be directly sampled from X0 using a closed-form expression: X_t = sqrt(alpha_bar_t) * X0 + sqrt(1 - alpha_bar_t) * epsilon, where alpha_bar_t is a product of (1 - beta_i) terms. As t approaches infinity, alpha_bar_t approaches 0, meaning X_t converges to a standard Gaussian distribution, regardless of the initial X0.
The reverse diffusion process: learning to denoise
The goal is to learn a reverse process that can generate a clean image X0 starting from pure noise XT. This reverse process is modeled as a Markov chain, where at each step t, we predict the slightly less noisy image X_{t-1} given the current noisy image X_t. This conditional distribution, P_theta(X_{t-1} | X_t), is parameterized as a Gaussian distribution. The mean of this Gaussian is predicted by a neural network (mu_theta) that takes X_t and the timestep t as input, while the variance (sigma_t^2) is often fixed or learned. The network's task is to learn the parameters theta of this mean predictor, effectively learning to reverse the noise addition step by step. This learned reverse process allows generation by starting with random Gaussian noise and iteratively applying the learned denoising function from T down to 0.
Training objective: maximizing the evidence lower bound
The training objective for diffusion models is typically derived from the Evidence Lower Bound (ELBO) of the data likelihood, similar to Variational Autoencoders. The ELBO provides a tractable lower bound on the log-likelihood of the data. In the context of diffusion models, this bound can be decomposed using the chain rule for KL divergence. The overall objective simplifies into a sum of per-step objective functions, each corresponding to a single denoising step. This decomposition makes the training process more manageable, as each step can be optimized independently.
Simplification to a denoising score matching objective
When the ELBO is decomposed and simplified, the training objective for diffusion models effectively becomes a form of denoising score matching. The primary term in the loss function involves minimizing the difference between the noise predicted by the neural network and the actual noise that was added during the forward process at each timestep. Specifically, the loss can be expressed as minimizing the KL divergence between the true reverse transition (conditioned on X0) and the parameterized reverse transition P_theta. Due to the properties of Gaussian distributions and the structure of the forward process, this KL divergence, when computed, primarily depends on the difference between the predicted mean and the true mean, leading to a loss function that encourages the network to accurately predict the noise component at each step. This is often implemented by training the network to predict the noise epsilon itself, scaled by a factor related to beta_t.
Theoretical underpinnings: continuous-time perspective
The choice of parameterizing the reverse process as a Gaussian distribution is theoretically justified by considering the continuous-time limit of the diffusion process. Using stochastic differential equations (SDEs), it can be shown that the reverse diffusion process, under certain conditions, is also Gaussian. This deep result, with roots in stochastic process theory dating back to the 1940s and formalized in later work like Anderson (1985), provides a strong rationale for the discrete-time Gaussian parameterization used in practice. It suggests that as the number of diffusion steps increases and step sizes become infinitesimally small, the learned Gaussian reverse step accurately approximates the true, continuous reverse process.
Mentioned in This Episode
●Software & Apps
●Studies Cited
●Concepts
●People Referenced
Common Questions
Diffusion models are a type of generative model primarily used for creating realistic images, but also applicable to videos and robotics. They work by gradually adding noise to data and then learning to reverse this process to generate new samples.
Topics
Mentioned in this video
An iterative algorithm used for finding maximum likelihood estimates of parameters in statistical models, particularly relevant to diffusion model training.
A type of noise characterized by a normal distribution, fundamental to the diffusion process for both adding and reversing noise.
A fundamental concept in stochastic processes, representing random movement, which underpins the continuous-time diffusion models.
A set of principles in probability theory dealing with updating beliefs based on evidence, used in deriving diffusion model properties.
Kullback-Leibler divergence, a measure of how one probability distribution diverges from a second, used as a regularization term in the diffusion model loss function.
Generative Adversarial Networks, a class of machine learning frameworks used for generative modeling, which diffusion models have largely surpassed.
A type of generative model that learns a latent representation of data, often used before diffusion models became predominant.
Multi-layer perceptron, a type of artificial neural network often used as a component in diffusion models.
More from Stanford Online
View all 106 summaries
77 minStanford CS229 Machine Learning | Spring 2026 | Lecture 18: GMM (EM), PCA
57 minStanford CS547 HCI Seminar | Spring 2026 | Show It or Tell It? Text, Visualization, and Combination
74 minStanford CS229 Machine Learning | Spring 2026 | Lecture 16: Basic Concept in RL, Policy Gradient
81 minStanford CS229 Machine Learning | Spring 2026 | Lecture 10: GMM (EM), PCA
Ask anything from this episode.
Save it, chat with it, and connect it to Claude or ChatGPT. Get cited answers from the actual content — and build your own knowledge base of every podcast and video you care about.
Get Started Free