Attention Mechanism

Concept

An improvement over encoder-decoder architectures that allows the model to look back at specific parts of the input sequence during decoding, improving translation accuracy.

Mentioned in 2 videos

Videos Mentioning Attention Mechanism

Deep Learning State of the Art (2019)

Deep Learning State of the Art (2019)

Lex Fridman

An improvement over encoder-decoder architectures that allows the model to look back at specific parts of the input sequence during decoding, improving translation accuracy.

Torch Tutorial (Alex Wiltschko, Twitter)

Torch Tutorial (Alex Wiltschko, Twitter)

Lex Fridman

A mechanism used in neural networks to allow the model to focus on specific parts of the input sequence, particularly useful in NLP and sequence-to-sequence tasks.