Tensor Parallelism

A Comprehensive Overview of Large Language Models - Latent Space Paper Club

Latent Space

A distributed training technique that splits model tensors across multiple devices.