DeepSeek Sparse Attention

Concept

A method that selects a subset of KV cache tokens to keep, using lighter-weight queries to determine which tokens are important.

Mentioned in 1 video

Videos Mentioning DeepSeek Sparse Attention

Stanford CS336 Language Modeling from Scratch | Spring 2026 | Lecture 10: Inference

Stanford CS336 Language Modeling from Scratch | Spring 2026 | Lecture 10: Inference

Stanford Online

A method that selects a subset of KV cache tokens to keep, using lighter-weight queries to determine which tokens are important.