Adam optimizer

Concept

The optimization algorithm chosen for training in the lecture (recommended default for training transformers).

Mentioned in 2 videos

Videos Mentioning Adam optimizer

Let's build GPT: from scratch, in code, spelled out.

Let's build GPT: from scratch, in code, spelled out.

Andrej Karpathy

The optimization algorithm chosen for training in the lecture (recommended default for training transformers).

⚡ Open Model Pretraining Masterclass — Elie Bakouch, HuggingFace SmolLM 3, FineWeb, FinePDF

⚡ Open Model Pretraining Masterclass — Elie Bakouch, HuggingFace SmolLM 3, FineWeb, FinePDF

Latent Space