Yet Another Rope Extension

Concept

A technique for extending the context length of models by tweaking the frequency of rotary positional embeddings to handle longer sequences.

Mentioned in 1 video

Videos Mentioning Yet Another Rope Extension

OpenAI vs. Deepseek vs. Qwen: Comparing Open Source LLM Architectures

OpenAI vs. Deepseek vs. Qwen: Comparing Open Source LLM Architectures

Y Combinator

A technique for extending the context length of models by tweaking the frequency of rotary positional embeddings to handle longer sequences.