momentum

Concept

An optimization technique that helps accelerate gradient descent in the relevant direction and dampens oscillations by adding a fraction of the previous update to the current gradient.

Mentioned in 1 video

Videos Mentioning momentum

Foundations of Deep Learning (Hugo Larochelle, Twitter)

Foundations of Deep Learning (Hugo Larochelle, Twitter)

Lex Fridman

An optimization technique that helps accelerate gradient descent in the relevant direction and dampens oscillations by adding a fraction of the previous update to the current gradient.