All Reduce
Concept
A collective communication operation where data from all ranks is reduced (e.g., summed) and the result is distributed to all ranks.
Mentioned in 1 video
A collective communication operation where data from all ranks is reduced (e.g., summed) and the result is distributed to all ranks.
