LayerNorm

Concept

Layer normalization used in the Transformer; the lecture implements pre-norm LayerNorm to stabilize deep network training.

Mentioned in 1 video

Videos Mentioning LayerNorm

Let's build GPT: from scratch, in code, spelled out.

Let's build GPT: from scratch, in code, spelled out.

Andrej Karpathy

Layer normalization used in the Transformer; the lecture implements pre-norm LayerNorm to stabilize deep network training.