Sigmoid Attention
Study / ResearchMentioned in 1 video
A recent paper that studies the distribution of logits and attention weights, relevant to improving long context handling in language models.
A recent paper that studies the distribution of logits and attention weights, relevant to improving long context handling in language models.
![[Paper Club] Writing in the Margins: Chunked Prefill KV Caching for Long Context Retrieval](https://i.ytimg.com/vi/VHwrhL_MSV4/maxresdefault.jpg)