W

WSD schedule

ConceptMentioned in 1 video

A type of schedule that was compared against schedule-free optimizers, but the latter did not perform as well in testing.

Videos Mentioning WSD schedule

⚡ Open Model Pretraining Masterclass — Elie Bakouch, HuggingFace SmolLM 3, FineWeb, FinePDF

⚡ Open Model Pretraining Masterclass — Elie Bakouch, HuggingFace SmolLM 3, FineWeb, FinePDF

Latent Space

A type of schedule that was compared against schedule-free optimizers, but the latter did not perform as well in testing.