KV Cache
Concept
A caching mechanism for Transformers that stores previously computed keys and values, significantly speeding up token generation by avoiding redundant computations during subsequent passes.
Mentioned in 2 videos
Save the 2 videos on KV Cache to your own pod.
Sign up free to keep building your knowledge base on KV Cache as more episodes are added.
Videos Mentioning KV Cache
Ep 18: Petaflops to the People — with George Hotz of tinycorp
Latent Space
The KV cache invalidation problem with large context windows is mentioned as a drawback of some positional embedding techniques.
Cursor Team: Future of Programming with AI | Lex Fridman Podcast #447
Lex Fridman
A caching mechanism for Transformers that stores previously computed keys and values, significantly speeding up token generation by avoiding redundant computations during subsequent passes.