SGD
gradient descent method used for the minimization of an objective function
What podcasters actually say about SGD.
7 mentions, no marketing. Save them all to a pod and ask any question.
Common Themes
Videos Mentioning SGD
Building makemore Part 3: Activations & Gradients, BatchNorm
Andrej Karpathy
Stochastic gradient descent (baseline optimizer) used in examples; compared to Adam/RMSprop when discussing practical stability.
Breaking down the OG GPT Paper by Alec Radford
Latent Space
Mentioned as an example of an optimizer relevant to word embeddings.
MIT 6.S094: Recurrent Neural Networks for Steering Through Time
Lex Fridman
Stochastic Gradient Descent, a common optimization algorithm for training neural networks, discussed as the vanilla approach that can find solutions despite non-convex landscapes.
Stanford CS336 Language Modeling from Scratch | Spring 2026 | Lecture 2: PyTorch (einops)
Stanford Online
Stochastic Gradient Descent, mentioned as a foundational optimization algorithm from which Adagrad and Adam evolved.
Stanford CS336 Language Modeling from Scratch | Spring 2026 | Lecture 9: Scaling Laws
Stanford Online
Stochastic Gradient Descent, an optimization algorithm discussed in the context of scaling laws and compared against Adam, highlighting similar scaling trends despite different methodologies.
Stanford CS229 Machine Learning | Spring 2026 | Lecture 14: Transformers, In-Context Learning
Stanford Online
Mentioned as an optimizer used for training, alongside its extension, IDOM.
Stanford CS229 Machine Learning | Spring 2026 | Lecture 3: Weighted Least Squares
Stanford Online
A foundational optimization method discussed previously, used here in the context of solving for model parameters.