softmax / cross-entropy gradient (analytic formula)

Tool / ProductMentioned in 1 video

Analytically derived gradient of cross-entropy w.r.t. logits: p - 1_{y} (scaled by 1/N for batch).

Videos Mentioning softmax / cross-entropy gradient (analytic formula)

Building makemore Part 4: Becoming a Backprop Ninja

Building makemore Part 4: Becoming a Backprop Ninja

Andrej Karpathy

Analytically derived gradient of cross-entropy w.r.t. logits: p - 1_{y} (scaled by 1/N for batch).