Transformer

Software / App

A type of neural network architecture mentioned in the context of reasoning and working memory, with limitations related to recurrence and fixed layers.

Mentioned in 3 videos

Videos Mentioning Transformer

Yann LeCun: Deep Learning, ConvNets, and Self-Supervised Learning | Lex Fridman Podcast #36

Yann LeCun: Deep Learning, ConvNets, and Self-Supervised Learning | Lex Fridman Podcast #36

Lex Fridman

A type of neural network architecture mentioned in the context of reasoning and working memory, with limitations related to recurrence and fixed layers.

Oriol Vinyals: DeepMind AlphaStar, StarCraft, and Language | Lex Fridman Podcast #20

Oriol Vinyals: DeepMind AlphaStar, StarCraft, and Language | Lex Fridman Podcast #20

Lex Fridman

A neural network architecture, very popular in natural language processing since 2017, and also used in AlphaStar to integrate past observations and actions.

Deep Learning State of the Art (2019)

Deep Learning State of the Art (2019)

Lex Fridman

A model architecture that utilizes self-attention in the encoder and attention in the decoder to capture rich context from the input sequence for output generation.