Switch Transformer

Software / App

An early Google paper on MoEs that introduced the concept of auxiliary loss for expert balancing.

Mentioned in 1 video

Videos Mentioning Switch Transformer

Stanford CS336 Language Modeling from Scratch | Spring 2026 | Lecture 4: Attention Alternatives

Stanford CS336 Language Modeling from Scratch | Spring 2026 | Lecture 4: Attention Alternatives

Stanford Online

An early Google paper on MoEs that introduced the concept of auxiliary loss for expert balancing.