N

NVIDIA 2018 online softmax (paper)

Study / ResearchMentioned in 1 video

Earlier NVIDIA paper describing an online softmax normalization technique reused by FlashAttention.

Videos Mentioning NVIDIA 2018 online softmax (paper)

Let's reproduce GPT-2 (124M)

Let's reproduce GPT-2 (124M)

Andrej Karpathy

Earlier NVIDIA paper describing an online softmax normalization technique reused by FlashAttention.