attention
Concept
A mechanism in neural networks that allows the system to selectively focus on specific parts of the input data during calculation, improving performance and interpretability.
Mentioned in 3 videos
Videos Mentioning attention
A Comprehensive Overview of Large Language Models - Latent Space Paper Club
Latent Space
A mechanism in neural networks that weighs the importance of different parts of the input sequence, crucial before Transformers.
AI Language Models & Transformers - Computerphile
Computerphile
A mechanism in neural networks that allows the system to selectively focus on specific parts of the input data during calculation, improving performance and interpretability.
Transformers Explained: The Discovery That Changed AI Forever
Y Combinator
A mechanism that allows neural networks, particularly decoders in sequence-to-sequence models, to focus on relevant parts of the input sequence when generating an output.