Batch Normalization
normalization technique used to make training faster and more stable by adjusting the inputs to each layer, recentering them around zero and rescaling them to a standard size
Save the 4 videos on Batch Normalization to your own pod.
Sign up free to keep building your knowledge base on Batch Normalization as more episodes are added.
Videos Mentioning Batch Normalization
Ep 18: Petaflops to the People — with George Hotz of tinycorp
Latent Space
Batch Normalization is listed as one of the 'six tricks' in AI development, with George Hotz wondering if phenomena like 'covaria shift' inspired it.
Jeremy Howard: fast.ai Deep Learning Courses and Research | Lex Fridman Podcast #35
Lex Fridman
A technique for improving the training of deep neural networks, cited as a major breakthrough that didn't require multiple GPUs.
Deep Learning for Speech Recognition (Adam Coates, Baidu)
Lex Fridman
An extremely helpful strategy for training recurrent and very deep neural networks, widely popular and available as an off-the-shelf package in many deep learning frameworks.
Foundations of Deep Learning (Hugo Larochelle, Twitter)
Lex Fridman
A technique that normalizes the activations of hidden layers during training, which can improve optimization speed, stability, and act as a regularizer.