Q

QK norm

Tool / ProductMentioned in 1 video

Another method mentioned that aims to patch the same issue as QK clip, but QK clip is considered more elegant.

Videos Mentioning QK norm

⚡ Open Model Pretraining Masterclass — Elie Bakouch, HuggingFace SmolLM 3, FineWeb, FinePDF

⚡ Open Model Pretraining Masterclass — Elie Bakouch, HuggingFace SmolLM 3, FineWeb, FinePDF

Latent Space

Another method mentioned that aims to patch the same issue as QK clip, but QK clip is considered more elegant.