FlashAttention

Concept

An efficient kernel for running the attention operation inside the Transformer architecture, developed in academia.

Mentioned in 1 video

Videos Mentioning FlashAttention

Andrej Karpathy: Tesla AI, Self-Driving, Optimus, Aliens, and AGI | Lex Fridman Podcast #333

Andrej Karpathy: Tesla AI, Self-Driving, Optimus, Aliens, and AGI | Lex Fridman Podcast #333

Lex Fridman

An efficient kernel for running the attention operation inside the Transformer architecture, developed in academia.