softmax

Concept

The neural-network layer that exponentiates logits and normalizes them into a probability distribution over next characters.

Mentioned in 4 videos

Videos Mentioning softmax

The spelled-out intro to language modeling: building makemore

The spelled-out intro to language modeling: building makemore

Andrej Karpathy

The neural-network layer that exponentiates logits and normalizes them into a probability distribution over next characters.

Elon Musk on DOGE, Optimus, Starlink Smartphones, Evolving with AI, Why the West is Imploding

Elon Musk on DOGE, Optimus, Starlink Smartphones, Evolving with AI, Why the West is Imploding

All-In Podcast

Theano Tutorial (Pascal Lamblin, MILA)

Theano Tutorial (Pascal Lamblin, MILA)

Lex Fridman

A function applied in Theano for output probabilities, with optimized versions for numerical stability.

Foundations of Deep Learning (Hugo Larochelle, Twitter)

Foundations of Deep Learning (Hugo Larochelle, Twitter)

Lex Fridman

An activation function used in the output layer for classification tasks, which converts outputs into a probability distribution, ensuring all outputs sum to 1.