NVIDIA 2018 online softmax (paper)

Study / Research

Earlier NVIDIA paper describing an online softmax normalization technique reused by FlashAttention.

Mentioned in 1 video

Videos Mentioning NVIDIA 2018 online softmax (paper)

Let's reproduce GPT-2 (124M)

Let's reproduce GPT-2 (124M)

Andrej Karpathy

Earlier NVIDIA paper describing an online softmax normalization technique reused by FlashAttention.