tanh (hyperbolic tangent)

Software / App

The activation function used in the MLP; its backward rule (1 - a^2) is derived and used in the manual backward pass.

Mentioned in 1 video

Videos Mentioning tanh (hyperbolic tangent)

Building makemore Part 4: Becoming a Backprop Ninja

Building makemore Part 4: Becoming a Backprop Ninja

Andrej Karpathy

The activation function used in the MLP; its backward rule (1 - a^2) is derived and used in the manual backward pass.