FlashAttention
Software / AppMentioned in 1 video
Mentioned as an example of specialized kernels for Transformers, paralleled by efforts like FlashFFTConv for SSMs.
Mentioned as an example of specialized kernels for Transformers, paralleled by efforts like FlashFFTConv for SSMs.
![2024 in Post-Transformer Architectures: State Space Models, RWKV [Latent Space LIVE! @ NeurIPS 2024]](https://i.ytimg.com/vi/LPe6iC73lrc/maxresdefault.jpg)