Nesterov Acceleration

Concept

A surprising and deep idea in optimization that improves gradient descent's convergence rate, often by introducing momentum.

Mentioned in 1 video