tanh (hyperbolic tangent)

Tool / ProductMentioned in 1 video

The activation function used in the MLP; its backward rule (1 - a^2) is derived and used in the manual backward pass.

Videos Mentioning tanh (hyperbolic tangent)

Building makemore Part 4: Becoming a Backprop Ninja

Building makemore Part 4: Becoming a Backprop Ninja

Andrej Karpathy

The activation function used in the MLP; its backward rule (1 - a^2) is derived and used in the manual backward pass.