Group query attention
Concept
An efficient attention scheme used in Transformer models that reduces the size of the KV cache by using fewer heads for keys and values while preserving query heads, improving inference speed.
Mentioned in 2 videos
Save the 2 videos on Group query attention to your own pod.
Sign up free to keep building your knowledge base on Group query attention as more episodes are added.
Videos Mentioning Group query attention
Cursor Team: Future of Programming with AI | Lex Fridman Podcast #447
Lex Fridman
An efficient attention scheme used in Transformer models that reduces the size of the KV cache by using fewer heads for keys and values while preserving query heads, improving inference speed.
Stanford CME 295 Transformers & LLMs | Fall 2026 | Lecture 3 - LLM training
Stanford Online
وسيلة لتوفير المفاتيح والقيم اللازمة لتشفير التسلسل.