GELU

Let's build GPT: from scratch, in code, spelled out.

Andrej Karpathy

A nonlinearity used in the Multilayer Perceptron (MLP) / feed-forward block (noted because OpenAI uses it).