Scatter

Stanford CS336 Language Modeling from Scratch | Spring 2026 | Lecture 7: Parallelism

Stanford Online

A collective communication operation where a tensor at one rank is split and distributed to other ranks.