RoPE
ConceptMentioned in 2 videos
A position embedding method used in Transformer architectures, appreciated for its extrapolation properties in long context models.
Videos Mentioning RoPE
The 10,000x Yolo Researcher Metagame — with Yi Tay of Reka
Latent Space
A position embedding method used in Transformer architectures, appreciated for its extrapolation properties in long context models.
A Comprehensive Overview of Large Language Models - Latent Space Paper Club
Latent Space
A method for positional encoding, likely a more advanced or efficient variant.