Multi-query attention
Concept
The most aggressive efficient attention scheme in Transformer models, which uses only one key-value head, significantly reducing KV cache size and improving inference speed for larger batch sizes.
Mentioned in 2 videos
Save the 2 videos on Multi-query attention to your own pod.
Sign up free to keep building your knowledge base on Multi-query attention as more episodes are added.
Videos Mentioning Multi-query attention
Cursor Team: Future of Programming with AI | Lex Fridman Podcast #447
Lex Fridman
The most aggressive efficient attention scheme in Transformer models, which uses only one key-value head, significantly reducing KV cache size and improving inference speed for larger batch sizes.
Stanford CS336 Language Modeling from Scratch | Spring 2026 | Lecture 10: Inference
Stanford Online
An attention variant with K=1, mentioned as being very fast but generally not used due to poor performance.