Reduce Scatter

Concept

A collective communication operation that performs a reduction on tensors across ranks and then scatters the results to different ranks.

Mentioned in 1 video

Videos Mentioning Reduce Scatter

Stanford CS336 Language Modeling from Scratch | Spring 2026 | Lecture 7: Parallelism

Stanford CS336 Language Modeling from Scratch | Spring 2026 | Lecture 7: Parallelism

Stanford Online

A collective communication operation that performs a reduction on tensors across ranks and then scatters the results to different ranks.