torch.distributed.tensor.parallel.parallelize

Software / App

A function or mechanism within PyTorch's distributed tensor parallelism library that facilitates combining different parallelism strategies.

Mentioned in 1 video

Videos Mentioning torch.distributed.tensor.parallel.parallelize

Stanford CS25: Transformers United V6 I The Ultra-Scale Talk: Scaling Training to Thousands of GPUs

Stanford CS25: Transformers United V6 I The Ultra-Scale Talk: Scaling Training to Thousands of GPUs

Stanford Online

A function or mechanism within PyTorch's distributed tensor parallelism library that facilitates combining different parallelism strategies.