KV Cache
Concept
A caching mechanism for Transformers that stores previously computed keys and values, significantly speeding up token generation by avoiding redundant computations during subsequent passes.
Mentioned in 2 videos
Videos Mentioning KV Cache
Ep 18: Petaflops to the People — with George Hotz of tinycorp
Latent Space
The KV cache invalidation problem with large context windows is mentioned as a drawback of some positional embedding techniques.
Cursor Team: Future of Programming with AI | Lex Fridman Podcast #447
Lex Fridman
A caching mechanism for Transformers that stores previously computed keys and values, significantly speeding up token generation by avoiding redundant computations during subsequent passes.