softmax / cross-entropy gradient (analytic formula)

Software / App

Analytically derived gradient of cross-entropy w.r.t. logits: p - 1_{y} (scaled by 1/N for batch).

Mentioned in 1 video

Videos Mentioning softmax / cross-entropy gradient (analytic formula)

Building makemore Part 4: Becoming a Backprop Ninja

Building makemore Part 4: Becoming a Backprop Ninja

Andrej Karpathy

Analytically derived gradient of cross-entropy w.r.t. logits: p - 1_{y} (scaled by 1/N for batch).