Megatron

[Paper Club] Weight Streaming on Wafer-Scale Clusters (w/ Sarah Chieng of Cerebras)

Latent Space

Mentioned as a system that utilizes data parallelism for training, contrasted with Cerebras' weight streaming approach.