Batch Normalization
Batch Normalization is listed as one of the 'six tricks' in AI development, with George Hotz wondering if phenomena like 'covaria shift' inspired it.
Videos Mentioning Batch Normalization
Ep 18: Petaflops to the People — with George Hotz of tinycorp
Latent Space
Batch Normalization is listed as one of the 'six tricks' in AI development, with George Hotz wondering if phenomena like 'covaria shift' inspired it.
Jeremy Howard: fast.ai Deep Learning Courses and Research | Lex Fridman Podcast #35
Lex Fridman
A technique for improving the training of deep neural networks, cited as a major breakthrough that didn't require multiple GPUs.
Deep Learning for Speech Recognition (Adam Coates, Baidu)
Lex Fridman
An extremely helpful strategy for training recurrent and very deep neural networks, widely popular and available as an off-the-shelf package in many deep learning frameworks.
Foundations of Deep Learning (Hugo Larochelle, Twitter)
Lex Fridman
A technique that normalizes the activations of hidden layers during training, which can improve optimization speed, stability, and act as a regularizer.