Yarn scaling

Concept

A technique applied during pre-training in GPT OSS to achieve a 131,000 token context window by scaling rotary positional embeddings.

Mentioned in 1 video

Videos Mentioning Yarn scaling

OpenAI vs. Deepseek vs. Qwen: Comparing Open Source LLM Architectures

OpenAI vs. Deepseek vs. Qwen: Comparing Open Source LLM Architectures

Y Combinator

A technique applied during pre-training in GPT OSS to achieve a 131,000 token context window by scaling rotary positional embeddings.