Mixed RoPE

Concept

A 2D position embedding strategy that mixes rotations with respect to both X and Y axes within the same vector, mitigating issues of lacking interaction between axes seen in Axial RoPE.

Mentioned in 1 video

Videos Mentioning Mixed RoPE

Stanford CME296 Diffusion & Large Vision Models | Spring 2026 | Lecture 5 - Architectures

Stanford CME296 Diffusion & Large Vision Models | Spring 2026 | Lecture 5 - Architectures

Stanford Online

A 2D position embedding strategy that mixes rotations with respect to both X and Y axes within the same vector, mitigating issues of lacking interaction between axes seen in Axial RoPE.