FlashAttention (paper)

Study / Research

Stanford paper and implementation fused into PyTorch kernels to speed up attention by avoiding materializing the full attention.

Mentioned in 1 video

Videos Mentioning FlashAttention (paper)

Let's reproduce GPT-2 (124M)

Let's reproduce GPT-2 (124M)

Andrej Karpathy

Stanford paper and implementation fused into PyTorch kernels to speed up attention by avoiding materializing the full attention.