WSD schedule

Concept

A type of schedule that was compared against schedule-free optimizers, but the latter did not perform as well in testing.

Mentioned in 1 video

Videos Mentioning WSD schedule

⚡ Open Model Pretraining Masterclass — Elie Bakouch, HuggingFace SmolLM 3, FineWeb, FinePDF

⚡ Open Model Pretraining Masterclass — Elie Bakouch, HuggingFace SmolLM 3, FineWeb, FinePDF

Latent Space

A type of schedule that was compared against schedule-free optimizers, but the latter did not perform as well in testing.