DDP

Concept

Distributed Data Parallelism, a basic form of distributed training where the model is replicated on each accelerator, and the data is sharded.

Mentioned in 1 video

Videos Mentioning DDP

AI Dev 25 x NYC | Robert Crowe: JAX Made Simple: An Intuitive Guide to Building Fast Neural Networks

AI Dev 25 x NYC | Robert Crowe: JAX Made Simple: An Intuitive Guide to Building Fast Neural Networks

DeepLearningAI

Distributed Data Parallelism, a basic form of distributed training where the model is replicated on each accelerator, and the data is sharded.