RMS Norm

ConceptMentioned in 1 video

A normalization technique used in Transformer architectures, noted as being non-controversial and a default choice in many models.

Videos Mentioning RMS Norm

The 10,000x Yolo Researcher Metagame — with Yi Tay of Reka

The 10,000x Yolo Researcher Metagame — with Yi Tay of Reka

Latent Space

A normalization technique used in Transformer architectures, noted as being non-controversial and a default choice in many models.