NCCL

Stanford CS336 Language Modeling from Scratch | Spring 2026 | Lecture 7: Parallelism

Stanford Online

NVIDIA's Collective Communications Library, which implements optimized collective operations for multi-GPU and multi-node communication.