LayerNorm

ConceptMentioned in 1 video

Layer normalization used in the Transformer; the lecture implements pre-norm LayerNorm to stabilize deep network training.

Videos Mentioning LayerNorm

Let's build GPT: from scratch, in code, spelled out.

Let's build GPT: from scratch, in code, spelled out.

Andrej Karpathy

Layer normalization used in the Transformer; the lecture implements pre-norm LayerNorm to stabilize deep network training.