Transformer
A type of neural network architecture mentioned in the context of reasoning and working memory, with limitations related to recurrence and fixed layers.
Save the 3 videos on Transformer to your own pod.
Sign up free to keep building your knowledge base on Transformer as more episodes are added.
Videos Mentioning Transformer
Yann LeCun: Deep Learning, ConvNets, and Self-Supervised Learning | Lex Fridman Podcast #36
Lex Fridman
A type of neural network architecture mentioned in the context of reasoning and working memory, with limitations related to recurrence and fixed layers.
Oriol Vinyals: DeepMind AlphaStar, StarCraft, and Language | Lex Fridman Podcast #20
Lex Fridman
A neural network architecture, very popular in natural language processing since 2017, and also used in AlphaStar to integrate past observations and actions.
Deep Learning State of the Art (2019)
Lex Fridman
A model architecture that utilizes self-attention in the encoder and attention in the decoder to capture rich context from the input sequence for output generation.