Multi-head attention
Concept
A component of Transformer models. Mentioned in contrast to more efficient attention schemes like group query and multi-query attention, which aim to reduce KV cache size.
Mentioned in 2 videos
Save the 2 videos on Multi-head attention to your own pod.
Sign up free to keep building your knowledge base on Multi-head attention as more episodes are added.
Videos Mentioning Multi-head attention
Cursor Team: Future of Programming with AI | Lex Fridman Podcast #447
Lex Fridman
A component of Transformer models. Mentioned in contrast to more efficient attention schemes like group query and multi-query attention, which aim to reduce KV cache size.
Stanford CME295 Transformers & LLMs | Autumn 2026 | Lecture 2 - Large Language Models
Stanford Online
آلية الانتباه المستخدمة في المحولات، حيث يتم إجراء عملية الانتباه عدة مرات بشكل متوازٍ.