RMS Norm

Concept

A normalization technique used in Transformer architectures, noted as being non-controversial and a default choice in many models.

Mentioned in 1 video

Videos Mentioning RMS Norm

The 10,000x Yolo Researcher Metagame — with Yi Tay of Reka

The 10,000x Yolo Researcher Metagame — with Yi Tay of Reka

Latent Space

A normalization technique used in Transformer architectures, noted as being non-controversial and a default choice in many models.