Sliding Window Attention
Concept
An attention mechanism that limits the context to the last k tokens, reducing KV cache size and making it suitable for long contexts.
Mentioned in 2 videos
Save the 2 videos on Sliding Window Attention to your own pod.
Sign up free to keep building your knowledge base on Sliding Window Attention as more episodes are added.
Videos Mentioning Sliding Window Attention
Stanford CME295 Transformers & LLMs | Autumn 2026 | Lecture 2 - Large Language Models
Stanford Online
تقنية انتباه تحد من نطاق التفاعل بين الرموز إلى نافذة معينة، مما يقلل من المتطلبات الحسابية.
Stanford CS336 Language Modeling from Scratch | Spring 2026 | Lecture 10: Inference
Stanford Online
An attention mechanism that limits the context to the last k tokens, reducing KV cache size and making it suitable for long contexts.