torch.autocast

Software / AppMentioned in 1 video

PyTorch autocasting context manager for mixed precision (bfloat16) training to reduce memory and increase speed.

Videos Mentioning torch.autocast

Let's reproduce GPT-2 (124M)

Let's reproduce GPT-2 (124M)

Andrej Karpathy

PyTorch autocasting context manager for mixed precision (bfloat16) training to reduce memory and increase speed.