VM
Software / AppMentioned in 1 video
An inference engine mentioned as having an experimental feature for chunked prefill, and using paged attention for KV cache allocation.
An inference engine mentioned as having an experimental feature for chunked prefill, and using paged attention for KV cache allocation.
![[Paper Club] Writing in the Margins: Chunked Prefill KV Caching for Long Context Retrieval](https://i.ytimg.com/vi/VHwrhL_MSV4/maxresdefault.jpg)