momentum

Foundations of Deep Learning (Hugo Larochelle, Twitter)

Lex Fridman

An optimization technique that helps accelerate gradient descent in the relevant direction and dampens oscillations by adding a fraction of the previous update to the current gradient.